Commit graph

1582 commits

Author SHA1 Message Date
Galen Charlton
26cdaaae77 Bug 10777: (follow-up) improve CSS3 for Zebra-striping report email output
This patch improves the CSS used to attempt to Zebra-stripe the
output of emailed reports.  This will work with some email clients,
but other email clients (e.g., Gmail) don't handle style elements in the
body or head element.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-03-11 14:57:26 +00:00
Nicholas van Oudtshoorn
25076dec54 Bug 10777: email HTML reports with the proper Content-Type
The misc/cronjobs/runreport.pl allows for sending html reports
via email. The problem is that the Content-Type isn't set to
text/html, which means that the generated html email isn't
displayed properly.

This patch set the Content-Type, and also adds a tiny bit of
CSS to potentially alternate row colours (just to make long
reports a bit easier on the eye!)

TEST PLAN
----------
1. Run the script similar to this:
    ./misc/cronjobs/runreport.pl --format=html --to=YOUREMAIL --subject="Bad Formatting!" REPORTNUMBER
2. Look at the email - the html code should by visible and ugly.
3. apply the patch
4. Run the script again.
5. Look at the email - the data should look nicer now.

Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-03-11 14:57:23 +00:00
Jonathan Druart
4ae3665ad8 Bug 10611: Use mysql_auto_reconnect instead of ping
DBD::Mysql provides a mysql_auto_reconnect flag. Using it avoids
the time required to do a $dbh->ping().

Benchmarks:

use Modern::Perl;
use C4::Context;
for ( 1 .. 1000 ) {
    $dbh = C4::Context->dbh;
}

* without this patch on a local DB:
perl t.pl  0,49s user 0,02s system 98% cpu 0,525 total
* without this patch on a remote DB:
perl t.pl  0,52s user 0,05s system 1% cpu 37,358 total
* with this patch on a local DB:
perl t.pl  0,46s user 0,04s system 99% cpu 0,509 total
* with this patch on a remote DB:
perl t.pl  0,49s user 0,02s system 56% cpu 0,892 total

Testing the auto reconnect:
use Modern::Perl;
use C4::Context;
my $ping = $dbh->ping;
say $ping;
$dbh->disconnect;
$ping = $dbh->ping;
say $ping;

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Comment: Real improvement. No koha-qa errors

prove t/db_dependent/Circulation_issuingrules.t produces no error
prove t/db_dependent/Context.t produces no error

Test
1) dumped Koha DB, load it on a non-local server
2) run sample script whit and without patch, local and remote

use Modern::Perl;
use C4::Context;
for ( 1 .. 100000 ) {
    my $dbh = C4::Context->dbh;
}

Main difference I note is with remote server
a) without patch
real    0m16.357s
user    0m2.592s
sys     0m2.132s

b) with patch
real    0m0.259s
user    0m0.240s
sys     0m0.012s

I think this could be good for DBs placed on
remote servers

Bug 10611: add a "new" parameter to C4::Context->dbh

When dbh->disconnect is called and the mysql_auto_reconnect flag is set,
the dbh is not recreated: the old one is used.

Adding a new flag, we can now force the C4::Context->dbh method to
return a new dbh.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>

Bug 10611: Followup: remove useless calls to dbh->disconnect

These 3 calls to disconnect are done at the end of the script, they are
useless.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-03-10 23:15:35 +00:00
Galen Charlton
03338b70e4 Bug 10955: (follow-up) improve usage information
This patch improves rebuild_zebra.pl's usage help
by explaining when --skip-deletes should be considered
and noting that it should be used in conjunction with
a cronjob to process deletions after hours.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-03-10 18:46:28 +00:00
b0870311e1 Bug 10955 - Add ability to skip deletions in zebraqueue
It seems that record deletions can cause extreme slowdowns for Koha
installations with extremely large numbers of records. It would be
helpful to be able to skip record deletions when processing the
zebraqueue with rebuild_zebra.pl so the deletions can be processed with
a lower frequency.

Test Plan:
1) Disable any zebra indexing cronjobs you may have
2) Delete a record
3) Note the operation recordDelete in the zebraqueue table having done = 0
4) Run misc/migration_tools/rebuild_zebra.pl -b -z --skip-deletes
5) Note the delete still has done = 0
6) Run misc/migration_tools/rebuild_zebra.pl -b -z
7) Note the delete now has done = 1

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.
Also tested for authorities, no problems found.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

RM note: this is at best a work-around, and I will emphasize that
--skip-deletes should be used only when absolutely necessary.

I hope that --skip-deletes can go away at some point soon, but
that may depend on changes to Zebra.
2014-03-10 18:44:10 +00:00
Galen Charlton
160c44d4e9 Bug 11078: (follow-up) tidy code
- fix a couple typos in comments
- make replace a "$i" with a more descriptive variable name
- style some of the new code

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-02-28 22:24:28 +00:00
07de37f0e5 Bug 11078: QA Follow-up for missing file permissions on lockfile
The original patch creates a lockfile in the ZEBRA_LOCKDIR.
It can fall back to /var/lock or even /tmp.
If the create fails, it dies. This can be considered as very
exceptional.

This followup adjusts the fallback location in /var/lock or /tmp
slightly.  It appends the database name to the folder in order to
prevent interfering between multiple Koha instances. Creation of the
lockfile has been moved to a subroutine extending directory and file
creation testing.

In the very unlikely case that we cannot create the lockfile (after
three separate tries), this follow-up allows you to continue instead
of die.  This is just as we did before we had file locking here. Every
time skipping a reindex could cause more harm than continuing and
having the race condition once in a while.

Test plan:
Test adding and removing lockdir from your koha-conf.xml. Check fallback.
Note that fallback in /var/lock or /tmp must contain database name.
Remove the lockdir config line and remove permissions from fallback. In
this case the reindex should continue but with a warning.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Tested with daemon and one-off invocation simultaneously.
Tested new wait parameter.
Tried all variations of lock directory (changing permissions etc.)

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-02-28 22:22:47 +00:00
Doug Kingston
88e7faf860 Bug 11078: Add locking to rebuild_zebra
This patch adds locking to rebuild_zebra.pl to ensure that simultaneous
changes are prevented (as one is likely to overwrite the other).
Incremental updates in daemon mode will skipped if the lock is busy
and they will be picked up on the next pass.  Non-daemon mode
invocations will also exit immediately if they cannot get the lock
unless the new flag -wait-for-lock is specified, in which case they
will wait until the get the lock and then proceed.

Supporting changes made to Makefile.PL and templates for the new
locking directory (paralleling the other zebra lock directories).
We stash the zebra_lockdir in koha-conf.xml so rebuild_zebra.pl
can find it.

To address earlier QA concerns we:
1. added code to check if flock is available and ignore locking if
it's missing (from M. de Rooy)

2. changed default for adhoc invocations to abort if they cannot
obtain the lock.  Added option -wait-for-lock if the user prefers
to wait until the lock is free, and then continue processing.

3. added missing entry to t/db_dependent/zebra_config.pl

4. added a fallback locking directory of /tmp

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Doug merged the original patch with the QA changes.
Just for the record, noting here that the original patch was tested
extensively too by Martin Renvoize.
I have added a followup for some exceptional cases.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-02-28 22:21:41 +00:00
daf2ebc4f5 Bug 11096: support the retrieval of large MARCXML records
This patch makes Koha <-> Zebra use MARCXML for the serialization when
using DOM, and USMARC for GRS-1.

* The following functions are modified to set the Zebra record syntax
according to the current sysprefs and configuration:

- C4::Context->Zconn
- C4::Context-_new_Zconn

* A new function 'new_record_from_zebra' is introduced, which checks the
context we are in, and creates the MARC::Record object using the right
constructor.

The following packages get touched to make use of the new function:
- C4::Search
- C4::AuthoritiesMarc

and the same happens to the UI scripts that make use of them (both in
the OPAC and STAFF interfaces).

* Calls to the unsafe ZOOM::Record->render()[1] method are removed.

Due to this last change the code for building facets was rewritten. And
for performance on the facets creation I pushed higher version
dependencies for MARC::File::XML and MARC::Record (we rely on
MARC::Field->as_string).

* Calls to MARC::Record->new_from_xml and MARC::Record->new_from_usmarc
are wrapped with eval for catching problems [2].

* As of bug 3087, UNIMARC uses the 'unimarc' record syntax. this case is
  correctly handled.
* As of bug 7818 misc/migration_tools/rebuild_zebra.pl behaves like:

- bib_index_mode (defaults to 'grs1' if not specified)
- auth_index_mode (defaults to 'dom')

here we do exactly the same.

To test:
 - prove t/db_dependent/Search.t should pass.
 - Searching should remain functional.
 - Indexing and searching for a big record should work (that's what the
   unit tests do).
 - Test an index scan search (on the staff interface):
    Search > More options > Check "Scan indexes".
 - Enable 'itemBarcodeFallbackSearch' and try to circulate any word, it
   shouldn't break.
 - Searching for a biblio in a new subscription shouldn't break.
 - Running bulkmarcimport.pl shouldn't break.
 - And so on... for the rest of the .pl files.

[1] http://search.cpan.org/~mirk/Net-Z3950-ZOOM/lib/ZOOM.pod#render()
[2] a record that cannot be parsed by MARC::Record is simply skipped (bug 10684)

Sponsored-by: Universidad Nacional de Cordoba
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-02-28 19:50:09 +00:00
Jonathan Druart
bab96c3f22 Bug 11571: fix breakage of -f option for translate script
Bug 7934 breaks the -f option introduced in bug 9161; this patch
repairs the regression.

While the generation of the command tmpl_process3.pl, a space is
missing if the -x option is given.

The tmpl_process3.pl is called like:
/home/koha/src/misc/translator/tmpl_process3.pl -q update -i
/home/koha/src/koha-tmpl/intranet-tmpl/prog/en/ -s
/home/koha/src/misc/translator/po/fr-FR-i-staff-t-prog-v-3006000.po -r
-x 'help'-f pay.tt

Revised test plan:
1) cd ./misc/translator
2) put a warn at LangInstaller.pm line 375.
3) time ./translate update fr-FR -f pay.tt
-- note the execution time and the output. The options in the
    command contain "-x 'help'-f pay.tt"
    The -f param is not passed to the script.
    The execution time is strangely long.
5) git reset --hard origin/master
6) apply this patch
7) put a warn at LangInstaller.pm line 375.
8) time ./translate update fr-FR -f pay.tt
-- verify the output and the execution time is now corrected.

Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-01-26 15:50:41 +00:00
Liz Rea
e6da56acb1 Bug 11138 - Add auto_unsuspend_reserves to crontab.example
To test:

Set up and run the cronjobs from crontab.example with a hold set to
unsuspend today. The hold should be unsuspended.

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-01-17 21:13:56 +00:00
Magnus Enger
e8a24c35f6 Bug 11188 - Make gather_print_notices.pl die on failed open()
Problem:
If you tell gather_print_notices.pl to write output to a location
you do not have write access to, it will silently fail to write the
data, but still mark unsent messages as sent.

Solution:
This patch adds two lines of defense:
1. Check that the location given for the output is writable
2. use "open() or die" instead of just "open()" when writing the
   output
The first measure should catch most of the potential errors, but
I guess a directory can be writable, but the open() still can fail
because the disk is full or something similar.

To test:
- Make sure you have some unsent messages in the message_queue table,
  that do not have an email adress
- Apply the patch
- Run the script, pointing at a location you do not have access to
  write to. Check that the script exits with an appropriate error
  message, and that the unsent messages are still unsent. Do this
  both with and without the -s option.
- To fake passing the first line of defence, comment out line 62
  and put this in instead:
  if ( !$output_directory || !-d $output_directory ) {
- Run the script again as above, check you get an appropriate
  error and that the message queue is not touched
- Reset line 62 to how it was
- Run the script against a directory you do have access to write to
  and check that output is produced as expected and that messages
  are marked as sent
- Sign off

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.
Works as described.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-12-30 15:12:48 +00:00
Matthias Meusburger
28d97e3228 Bug 11412: fix potential bulkmarcimport crash when searching for duplicates in authorities
bulkmarcimport.pl can crash when searching for duplicates if the 005
field from the incoming or local record is not defined. This patch
fixes it.

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>

Test plan
1/ Create a record with no 005 field
2/ Try to import it checking for duplicates, notice it crashes
3/ Try with a record with a 005 field, but the one in Koha missing
one, still crashes
4/ Apply patch
5/ No more crash

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.
Patch fixes the problem described for importing authorities
with the bulkmarcimport.pl when trying to match with existing
records.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-12-26 15:52:02 +00:00
Galen Charlton
b26870e53d Bug 11252: remove deprecated -munge-config switch from rebuild_zebra.pl
The -munge-config switch has been deprecated for years, and
trying to use it would either not work at all or, if it did "work",
almost certainly damage one's Zebra configuration for Koha.

This patch removes this switch.

To test:

[1] Run rebuild_zebra.pl and verify that no mention is made
    of -munge-config.
[2] Run rebuild_zebra.pl to index records in one's test database
    and verify that there are no regressions.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Removing a really dangerous option

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.
Ran rebuild_zebra.pl with various options and confirmed
that data was reindexed successfully.
No regressions found.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-12-26 15:24:41 +00:00
Gaetan Boisson
6657860010 Bug 11417: make sure remove_unused_authorities.pl accepts --test
This patches adds support for the --test option, as well as a
short message telling the user the script is running in test mode.

Test plan :
- Launch the script with -h to see the help
- Launch the script with --test and --aut with an authtypecode
  that is used in your instance
- Make sure it does the same thing as launching it with -t
- Launch the script for real and make sure it still works as
  expected, deleting unused authorities.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-12-19 15:09:18 +00:00
Bernardo Gonzalez Kriegel
466a7320d1 Translation update - split staff UI and help files 2013-12-17 16:22:49 -03:00
Bernardo Gonzalez Kriegel
6cf065da5e Bug 7934: generate separate PO files for help pages
This patch modifies LangInstaller.pm to enable separation
of translation for Staff UI and Staff Help files.

It's a move that make sense, Help strings accounts for
44% of total word count for Staff, and as stated on
Comment #1, it could enable a different workflow on
translation work.

It's more a hack than an elegant solution, but it works.
Feel free to suggest another approach.

To test:

We need to test complete functionality, i.e. create and update
translation files and install translation, and verify that no
string is missing.

1) Before applying the patch, we need some data from staff file.
Pick your language, say de_DE, and

cd misc/translator/
perl translate update de-DE
msginit -i po/de-DE-i-staff-t-prog-v-3006000.po -o de-old.po --no-translator --no-wrap --locale=de_DE
egrep ^msgid de-old.po | sort | tee s-old | wc -l > number-old

We have the file s-old with all strings, and the number of strings on number-old

2) Apply the patch

3) New help file is called de-DE-staff-help.po, so create one
cp'ing old staff on new help

cp po/de-DE-i-staff-t-prog-v-3006000.po po/de-DE-staff-help.po

4) Make a new update, and analize

perl translate update de-DE
msginit -i po/de-DE-i-staff-t-prog-v-3006000.po -o de-new.po --no-translator --no-wrap --locale=de_DE
msginit -i po/de-DE-staff-help.po -o de-help.po --no-translator --no-wrap --locale=de_DE
egrep ^msgid de-new.po | sort | tee s-new | wc -l > number-new
egrep ^msgid de-help.po | sort | tee s-help | wc -l > number-help
cat s-new s-help | sort | uniq | tee s-all | wc -l > number-all
cat s-new s-help | sort | uniq -d | tee s-dup | wc -l > number-dup

Ideally what we need to found is:

diff s-old s-all = zero lines (old strings vs new strings)

In my test I got one line, but it's a false positive (the string "• " is present on new staff)

On numbers,

number-old - number-new - nummber-help + number-dup = 0
or
number-old - number-all = 0

(in my test again I have 1 as result, same string. Also there are 137
repeated lines between new staff and help)

All this tells me that all string to translate are preserved

5) Install translation

perl translate install de-DE

Enable language on staff, and check that help files are translated

6) Finally, create translation files

rm po/de-DE-*
perl translate create de-DE

verify that all files are created. Tests of 4) can be repeated.

7) Verify that no strings from help are present on staff UI file

egrep help po/de-DE-i-staff-t-prog-v-3006000.po

Only results came from help-top and bottom, and a few "help" on staff strings

Signed-off-by: Fridolyn SOMERS <fridolyn.somers@biblibre.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Tested according to test plan. Checked various pages in OPAC,
staff and intranet, translation was ok.
Passes QA script and tests.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-12-17 05:42:20 +00:00
1219b3153e Bug 11124: QA Follow-up resolving a warning and three typos
Resolves warning on uninitialized author in split on line 128.
Just adds the same behavior for title on line 129 for completeness.
Fixes typo on occurrences and two other minor typos.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-12-13 19:49:09 +00:00
Jonathan Druart
4604f595d6 Bug 11124: disallow trying to run the staff benchmark script as the DB account
On step 6 (at least), the circulation and return page redirect to the
select branch page and nothing is done.

The script should die if the user used is the sql administrator account.

Test plan:
Suppose that the sql admin account is root/root and koha/koha a
superlibrarian account.

1/
perl misc/load_testing/benchmark_staff.pl --steps=6
--url=http://admin.koha.local/cgi-bin/koha/
--password="koha" --user="koha"
should produce:
  ...
  Step 6
  ...

2/
perl misc/load_testing/benchmark_staff.pl --steps=1
--url=http://admin.koha.local/cgi-bin/koha/
--password="root" --user="root"
should produce:
Authentication successful
You cannot use the database administrator account to launch this script

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-12-13 19:47:44 +00:00
Galen Charlton
b25de3e7cf Bug 6435: (follow-up) make -daemon really imply -a and -b
This patch follows up on the previous patch by moving the
check for whether authority and/or biblio indexing have been
specified so that -daemon has a chance to set those modes.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-11-24 18:20:56 +00:00
Doug Kingston
00240d6970 Bug 6435: (follow-up) rebuild_zebra -daemon option now smarter
Based on feedback, make daemon mode imply -z -a -b and abort
on startup if flags incompatible with an incremental update daemon
are used.  Update documentation to match.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-11-24 18:15:23 +00:00
Doug Kingston
1b0992e8d5 Bug 6435: Add daemon mode to rebuild_zebra.pl
This change adds code to check the zebraqueue table with a cheap SQL query
and a daemon loop that checks for new entries and processes them incrementally
before sleeping for a controllable number of seconds.  The default is 5 seconds
which provides a near realtime search index update.  This is desirable particularly
for libraries that are doing active catalogue updating.  The query is adjusted
based on whether -a, -b, or -a -b are specified.

Help text updated.  Tested against a live 3.12 system.

Note that this fix will benefit from the fix to lack of locking (bug 11078)

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-11-24 18:12:21 +00:00
Galen Charlton
07b5f01c07 release notes for Koha 3.14.0
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-11-21 01:40:18 +00:00
Bernardo Gonzalez Kriegel
83a5756645 Translation updates for 3.14.0
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-11-20 18:33:32 +00:00
0d21dc6c3b Bug 11185: fix bug preventing check-url-quick.pl from working
Revert commit f88f11b4f8

Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Before this patch, check-url-quick.pl fails with an error. After the
patch it runs correctly.

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-11-15 15:30:20 +00:00
Galen Charlton
a1d4af3d73 updated release notes for 3.14.0 beta
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-11-01 15:04:49 +00:00
Sophie Meynieux
394be84dcd Bug 10720: fix issue that prevented use of HTML tags in overdue notices
Substitution term in overdue notices shouldn't be removed if they don't match.
It prevents use of HTML in notices to use with CSS.

Test plan :
- define an overdue notice containing some valid substitution terms + some valid html tags (<strong> or <div class="my class" to use with a css
- find a borrower with overdues that trigger this notice
- run overdue_notices.pl -html <directory>

Without patch, your html tags are missing in your html file
With the patch, they are present and can be used with a css to generate a formatted pdf file.

If you add some invalid substitution term, with the patch, they are still in the final letter, but the warn in the log helps you to correct your notice template.

Signed-off-by: Srdjan <srdjan@catalyst.net.nz>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-11-01 00:28:04 +00:00
Galen Charlton
7a1c343c91 Bug 8015: (follow-up) update stage_file.pl
This patch fixes the call to BatchStageMarcRecords() made
by the comand-line record importer.  At some point in the future
we could go further and teach that tool how to apply modification
templates.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-31 22:51:13 +00:00
80897930b7 Bug 2720 - Overdues which debar automatically should undebar automatically when returned
This patch adds a more extensible and flexible debarments system to Koha. The fields
borrowers.debarred and borrowers.debarredcomment are retained for compatibility and
speed.

This system supports having debarments for multiple reasons. There are currently
three types of debarments:
OVERDUES    - Generated by overdue_notices.pl if the notice should debar a patron
SUSPENSION  - A punative debarment generated on checkin via an issuing rule
MANUAL      - A debarment created manually by a librarian

OVERDUE debarments are cleared automatically when all overdue items have been returned,
if the new system preference AutoRemoveOverduesRestrictions is enabled. It is disabled
by default to retain current default functionality.

Whenever a borrowers debarments are modified, the system updates the borrowers debarment
fields with the highest expiration from all the borrowers debarments, and concatenates
the comments from the debarments together.

Test plan:
 1) Apply patch
 2) Run updatedatabase.pl
 3) Verify the borrower_debarments table has been created and
    populated with the pre-existing debarments
 4) Run t/db_dependent/Borrower_Debarments.t
 5) Manually debar a patron, with an expiration date
 6) Verify the patron cannot be issued to
 7) Add another manual debarment with a different expiration date
 8) Verify the 'restricted' message lists the date farthest into the future
 9) Add another manual debarment with no expiration date
10) Verify the borrower is now debarred indefinitely
11) Delete the indefinite debarment
12) Verify the debarment message lists an expiration date dagain
13) Enable the new system preference AutoRemoveOverduesRestrictions
14) Set an overdue notice to debar after 1 day of being overdue
15) Check out an item to a patron and backdate the due date to yesterday
16) Run overdue_notices.pl
17) Verify the OVERDUES debarment was created
18) Return the item
19) Verify the OVERDUES debarment was removed
20) Disable AutoRemoveOverduesRestrictions
21) Repeat steps 15 though 18, verify the OVERDUES debarment was *not* removed
22) Add issuing rules so that an overdue item causes a temporary debarment
23) Check out an item to a patron and backdate the due date by a year
24) Return the item
25) Verify the SUSPENSION debarment was added to the patron

Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-31 01:53:22 +00:00
Galen Charlton
997a502c22 Bug 7131: (follow-up) update the command-line import tools
misc/commit_file.pl now properly displays the number
of replaced and ignored items.

misc/stage_file.pl now has a new command-line switch --item-action:

  --item-action       action to take if --add-items is specifed;
                      choices are 'always_add',
                      'add_only_for_matches', 'add_only_for_new',
                      'ignore', or 'replace'

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-30 04:35:17 +00:00
Bernardo Gonzalez Kriegel
616db836b9 Translation files for 3.14.0-beta
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-29 14:24:34 +00:00
Fridolyn SOMERS
20022e5b58 Bug 9192: let UNIMARC_sync_date_created_with_marc_biblio.pl create fields if needed
Prior to this patch, this script would only modify the date
created and modified fields; this patch changes the behavior
so that the fields can be created if they're not already present
in the record.

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Mason James <mtj@kohaaloha.com>

Bug 9192: UNIMARC_sync_date_created_with_marc_biblio.pl field creation (followup)

Add tests :
if field < 10 it must not have a subfield
if field > 9 is must have a subfield

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Both patches applied, no koha-qa errors.

Bug 9192: followup fix typo

Fix typo on usage message.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-23 14:20:14 +00:00
Galen Charlton
0c219d3dd8 update 3.14 release notes for second alpha
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-22 21:08:45 +00:00
Katrin Fischer
dbc0a84720 Bug 9739: (follow-up) improve help text a bit
Changes only strings in the script: the help text and the
success message for updated serial issues.

Thx to Jared for proof reading!

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-20 18:57:19 +00:00
Bernardo Gonzalez Kriegel
cc795c75d9 Bug 9739: (follow-up) add missing comma
Adds a missing comma in select statement.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-20 18:56:57 +00:00
Fridolyn SOMERS
ed38698247 Bug 9739: (follow-up) cosmetic changes
Cosmetic changes:

* perltidy
* q{} for query
* list of columns is query instead of SELECT *

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Your cosmetic changes missed a comma, fixed in a followup

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-20 18:56:27 +00:00
Fridolyn SOMERS
3a1a94d4ef Bug 9739: more options for serialsUpdate.pl
Script serialsUpdate.pl must be set in crontab to run daily.
It checks serials that are late, modifies status and adds a note.

This patch adds some options :
--note : defined note, a defaut one is used if not defined
--no-note : disable automatic note
--verbose : used to control output
This patch also implements confirmation option "-c" who whas present but not used.

Test plan :
Run script on a database with late serials :
- serialsUpdate.pl -h
=> you get help text in output
- serialsUpdate.pl --man
=> you get full help
- serialsUpdate.pl -v
=> you get output of changed serials but database has not changed
- serialsUpdate.pl -c -v
=> you get output of changed serials and database has changed, with note "Automatically set to late"
- serialsUpdate.pl -c
=> database has changed without anything in output
- serialsUpdate.pl -c --note "LATE"
=> database has changed, with note "LATE"
- serialsUpdate.pl -c --no-note
=> database has changed with no note

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Works as described. No koha-qa errors.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.
New options behave as described.
There are some existing problems with this script that I have noted
directly on the bug report.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-20 18:56:09 +00:00
Galen Charlton
5e8abb1191 start 3.14 release notes -- first alpha
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-14 23:47:08 +00:00
Galen Charlton
e0b0ae2053 bug 8798: (follow-up) pass DB connection params to DBIC schema updater explicitly
This patch changes update_dbix_class_files.pl so that the connection
parameters to the reference database must be passed explicitly via
command-line switches, rather than grabbing them from the current Koha
context.

The purpose of this is to intentionally put up a roadblock to reduce
the chance that a developer (or release manager) accidentally updates
the schema files to include local testing cruft such as temporary
tables.

Usage is now as follows:

[1] Create an empty database
[2] Load the schema creation script into it, e.g.,

mysql -u dbic -pdbic dbic < installer/data/mysql/kohastructure.sql

[3] Run the schema updater:

./misc/devel/update_dbix_class_files.pl --db_user=dbic --db_name=dbic --db_passwd=dbic

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-14 21:08:03 +00:00
bdf7988c46 Bug 8798: (follow-up) rename schema class updater script
updateDatabase.pl is a bit too close to updatedatabase.pl in installer
and may cause some confusion. I would suggest update_dbix_class_files.pl
as a unambiguous and descriptive name for this file.

Signed-off-by: wajasu <matted-34813@mypacks.net>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-14 21:08:01 +00:00
Elliott Davis
8a480ad75d Bug 8798: DBIx::Class base classes for all Koha tables
* Added base class files for all tables in koha using
DBIx::Class::Schema::Loader.
* Added a (very basic) test file for C4::Context
* Also added dependencies in required files.

To Test:

[1] Install patch
[2] Make sure you can still connect to Koha
[3] You may optionally run this test script:

        use Koha::Database;
        use Data::Dumper;
        my $db = Koha::Database->new();
        my $schema = $db->schema();
        print Dumper($schema->resultset("Borrower"));

    If you run this file you should get a DBIx dump of the borrowers table.

Signed-off-by: wajasu <matted-34813@mypacks.net>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-14 21:07:24 +00:00
Galen Charlton
88936cb524 bug 10419: (follow-up) improvements to the usage text
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-03 22:13:40 +00:00
Jonathan Druart
cf31604a0e Bug 10419: (follow-up) patrons with fines should not be deleted
Test plan:
- add a fine for a patron
- verify the script does not delete this patron.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Works, script do not delete a patron with fines.
No koha-qa errors with all patches applied

Test
1) Added a fine to a patron
2) run script
3) reports condition
Trying to delete patron 5... Failed to delete patron 5: patron has 10.00 in fines
4) Patron is not deleted

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.

Tested various combinations of options:
./delete_patrons.pl
  Gives a helpful message about the use of the script.
./delete_patrons.pl -h
  Outputs useful information about the use of the script
  and its various options.
./delete_patrons.pl --category_code ST --library CPL
  Gives the correct results in numbers and deletion was done
  properly.

Also tested:
  --expired_before
  --not_borrowed_since
  -v

Testing various conditions where a delete should not occur:
- Patron has checkouts
  Patron is not in list of patrons to delete (x patrons to delete)
- Patron has fines
  Patron is still in list of patrons to delete (x patrons to delete)

Checked deleted patrons had been moved to deletedborrowers.

Notes about possible enhancements:
- only print the success message 'x patrons deleted' when confirm
  flag was set
- patrons with current checkouts are silently excluded from the number
  of patrons to be deleted. Printing the number with current checkouts
  might be helpful.

Changes made:
  Fixed a small error in the documentation: expired_date is expired_before.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-03 21:44:53 +00:00
Jonathan Druart
b90e1ff892 Bug 10419: (follow-up) fix hardcoded records and pod
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Missing a sign. Removes harcoded values

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-03 21:44:21 +00:00
Galen Charlton
86c73e9b40 Bug 10419: (follow-up) functional improvements
[1] Patron deletion now happens atomically; if one part
    of the process fails, the record isn't left in a
    partially deleted state.
[2] The routine for handling lists properly during patron
    deletion is now invoked.
[3] The script now prints an indication if it's run
    without --confirm; otherwise, one might think that
    patron records were actually being deleted.
[4] --verbose now actually does something.

Without --verbose, the script will print the dry-run
warning (if applicable), the number of patrons to be
deleted, and error messages.

With --verbose, the script will also print a line with
the borrowernumber of each patron to be deleted.

To test:

[1] Run the script with and without --verbose and compare
    the, well, verbosity.
[2] Run the script without --confirm and note that the script
    prints a message saying that it's running in dry-run mode.
[3] Use the script to try to delete one or more patrons that have
    loans.  Confirm that error messages are printed reporting
    foreign constraints preventing the deletion.  Also confirm that
    no new rows are added to deletedborrowers for those patrons that
    could not be completely deleted.
[4] Use the script to delete a patron that has a public list.  Verify
    that after the deletion, the public list still exists but now
    has a null owner.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-03 21:42:03 +00:00
Galen Charlton
2a9c129d31 Bug 10419: (follow-up) tidy names of command-line switches
[1] Use --library instead of --branchcode to be consistent
    with other scripts and user-facing Koha terminology.
[2] Use --not_borrowed_since instead of --not_borrowered_since;
    no need to expose typos in the API to the user.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-03 21:41:50 +00:00
Chris Cormack
b8a16ee2fc Bug 10419: (followup) fix license statement
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-03 21:41:41 +00:00
Jonathan Druart
b9f74eaf41 Bug 10419: (follow-up) add branchcode parameter to patron deletion script
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-03 21:41:27 +00:00
Jonathan Druart
497795f32a Bug 10419: (followup) - tidy up cronjob for deleting patrons
This followup adds:
- execute flag (+x) for the cronjob script.
- replaces --dry-run with --confirm (according with existing scripts).
- changes output text (remove 'first person' style).
- updates the doc and simplifies the dates parameters.
- changes flags PrintError and RaiseError for the dbh in order to catch
  something if an error occurs...

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-03 21:41:14 +00:00
Jonathan Druart
231ed790e4 Bug 10419: new cronjob for deleting patrons
This patch provides a cronjob script for deleting patrons.

It takes 3 options:

--not_borrowed_since
--expired_date
--category_code

See the perldoc misc/cronjobs/delete_patrons.pl.
Use the -v flag for a verbose mode.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-03 21:33:04 +00:00