This patch adds locking to rebuild_zebra.pl to ensure that simultaneous
changes are prevented (as one is likely to overwrite the other).
Incremental updates in daemon mode will skipped if the lock is busy
and they will be picked up on the next pass. Non-daemon mode
invocations will also exit immediately if they cannot get the lock
unless the new flag -wait-for-lock is specified, in which case they
will wait until the get the lock and then proceed.
Supporting changes made to Makefile.PL and templates for the new
locking directory (paralleling the other zebra lock directories).
We stash the zebra_lockdir in koha-conf.xml so rebuild_zebra.pl
can find it.
To address earlier QA concerns we:
1. added code to check if flock is available and ignore locking if
it's missing (from M. de Rooy)
2. changed default for adhoc invocations to abort if they cannot
obtain the lock. Added option -wait-for-lock if the user prefers
to wait until the lock is free, and then continue processing.
3. added missing entry to t/db_dependent/zebra_config.pl
4. added a fallback locking directory of /tmp
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Doug merged the original patch with the QA changes.
Just for the record, noting here that the original patch was tested
extensively too by Martin Renvoize.
I have added a followup for some exceptional cases.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch use DataTable, see BUG|BZ 6836
- css/datatables.css
- lib/jquery/plugins/jquery.dataTables.min.js
- js/datatables.js
http://bugs.koha-community.org/show_bug.cgi?id=7167
Bug 7167 follow-up
Major changes:
* creating database tables for update on the fly, the 1st time the update script is called
* version is checked on mainpage.pl (and here only). If syspref Version differ from kohaversion.pl, the old updatedatabase is launched. If there are updates missing from new mechanism, the updatedatabase page is reached
* kohaversion check on each page is now useless in Auth.pm, removed dead code
* Updated installer: at the end of the process, retrieve all updates and automatically mark them "OK", as they're included in installer
Minor changes:
* adding copyright
* adding poddoc
* updating a warning, for better clarity
* switching from $$var to $var->
* small TT glitch fixed in updatedatabase.tt
* about.pl now returns the Version systempreference PLUS all the patches that have been applied
Bug 7167 follow-up perlcritic & numbers display & partial apply depending on DEBUG
* add use strict to updatedatabase, that is now perlcritic compliant
* partial apply of DB revs is now managed by DEBUG env variable = if DEBUG=0, the user can just apply every DBrev. If DEBUG=1, we're in a dev env, the user know has the option to apply DBrevs one by one
Display:
* in updatedatabase, small spelling changes
* in about.pl, remove 0 just after . (3.06.01 is displayed as 3.6.1)
* improve the display of applied numbers on about.pl
- before this patch, if you have N, N+1, N+2, N+3 and N+10 DB rev applied, about was displaying : , N+1 / N+2 / N+3 / N+10
- after this patch you have N......N+3 / N+10
* add ORDER BY into list_versions_already_knows to have number retrieved in the same order whatever the order they are applied
http://bugs.koha-community.org/show_bug.cgi?id=6679
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Bug 7167: Improve the update.pl script
* Added CLI options to update.pl
* Call update.pl from the installer.
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Bug 7167: Now, we check versions on mainpage.pl and after login
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Bug 7167: Reimplementing Marcel's suggestions & fixes
* Fixing the bugguy old version check (that was made against 3.0900000 instead of 3.0900027 -the last current kohaversion number
* in the CLI script, if there is nothing to report, just say it
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Bug 7167: Remove check_coherency
As suggested by Katrin, we've removed the call to check_coherency. It intended to provide readable comments when some SQL was wrong. Removing this sub result in the SQL error being displayed. That's OK because the sysadmin or the developer can google the error, understand it, then fix it.
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Bug 7167: Changing in .sql parsing
We first split on delimiter and then extract comments. You can now put
\n for delimiter comments.
ex:
DELIMITER ;
-- this is a comment
SELECT * FROM my_table;
-- another comment
Before this patch, we had to write:
DELIMITER ;
-- this is a comment;
SELECT * FROM my_table;
-- another comment;
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Bug 7167: Add .pl and .sql examples
Those files are in version directory, so will never be executed by the updater
If you want to provide an update, do it in a 3.09/ directory (if your update is expected for 3.10 version)
Note that the updater use a md5sum checker. So, if the same update is in 2 different places, it will be detected. That will be handy for changes made on both stable and master: a library running stable will get the update when updating. When upgrading to the next major release, Koha will detect the patch has already been applied, and no error will be thrown. With the previous mechanism, a DBRev ported to stable was re-executed when upgrading to master, resulting in a nasty (but usually harmless) error message
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Bug 7167: Improve display + factorize get_queries
Despite it's size, this patch is dealing with display questions only:
* The text "comments" and "queries" was hardcoded in ajax-updatedb-getinfo.pl script. It has been replaced by a JSON call, returning 2 separate values, "comments:" and "queries:" is now in the template, making it translatable
* Some minor tweak in the display (like putting things in bold, displaying OK in green, warnings in yellow and KO in red)
* Reordering the column headers for more readability:
* Status column is merged with availability, column is after status
* Status/availability terms more clear: "Not applied" instead of "unknown", "Applied and OK", "Applied and failed", "Applied and forced" are the 3 other statuses
* Removed one click to display comments on DBREv not yet applied: before the patch, one had to click "Show details", then "Get comments", now, "Get comments" is enough
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Bug 7167: FIX typos & moving a script to a proper place
* renamed availables to available
* renamed already_knows to already_applied
* fixed FSF & copyright headers
* removing a "use strict" because we already had use Modern::Perl
* fixed a tiny typo in about.tt
* moving update.pl to misc/bin because it's a CLI script
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Bug 7167: Add dependency File::Find::Rule
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Bug 7167: We want to execute non-numeric version with the -all option
Dealing with Marcel comment 100:
> Note that the current code around line 52/53 does not
> handle that correctly:
> Argument "\x{74}\x{65}..." isn't numeric in numeric ge (>=) at
> installer/data/mysql/update.pl line 52.
Now, a non-numeric DBRev will be applied if you provide the --all parameter, without throwing the error
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Bug 7167 reindentation & removing dead code
* The if (! defined $ENV{PERL5LIB}... block was wrongly intented
* The 3 lines running update.pl are useless: the update (new mechanism) is run from admin/updatedatabase.pl script. This part of install.pl is run only when you have "old style" DB revisions.
Summary:
* old mechanism = it's run as previously, by reaching the installer/install.pl?step=3 page, that applies all revisions
* new mechanism = when you log-in or reach mainpage.pl, you reach admin/updatedatabase.pl, where you can see what will be run, and run it
Tiny side effect = the check for old mechanism is now done *after* authentification (thus it's not done on each page call). It means that the user will have to enter login/password twice :
* first to log-in to Koha
* second to run installer/updatedatabase.pl?step=3
As the old mechanism is deprecated, we can expect this will happend only a few time in the history of a setup, it's not a big deal.
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Bug 7167: Don't raise an error in routine TableExists
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Bug 7167: FIX merge
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Bug 7167: Add .pl and .sql examples
Those files are in version directory, so will never be executed by the updater
If you want to provide an update, do it in a 3.09/ directory (if your update is expected for 3.10 version)
Note that the updater use a md5sum checker. So, if the same update is in 2 different places, it will be detected. That will be handy for changes made on both stable and master: a library running stable will get the update when updating. When upgrading to the next major release, Koha will detect the patch has already been applied, and no error will be thrown. With the previous mechanism, a DBRev ported to stable was re-executed when upgrading to master, resulting in a nasty (but usually harmless) error message
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Bug 7167 follow-up fix POD syntax to please koha-qa.pl
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
[This patch was split out from tcohen's excellent patches for bug 8519
--jcamins 2012/10/31]
Added a new koha-index-daemon-ctl.sh script that uses the
'koha-index-daemon' provided by Koha::Contrib::Tamil (install via CPAN)
to process the indexing queue ('zebraqueue' for now, 'indexqueue' in the
future I guess :-P).
This script could be easily modified in the future for using another
index queue processing script (a more abstract indexing script for both Solr
and Zebra, etc). I removed all zebra-ish stuff from it.
Documentation patches will be provided as followups, once Mark Tompsett's
docs are pushed to avoid continous rebasing.
Tested to work on an up-to-date Ubuntu 12.04.
Sponsored-by: Universidad Nacional de Córdoba
Signed-off-by: wajasu <matted-34813@mypacks.net>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
[This patch was split out from tcohen's excellent patches for bug 8519
--jcamins]
It removes the obsolete zebraqueue_daemon.pl and koha-zebraqueue-ctl.sh
obsolete scripts too.
Several files are modified to address te removal/addition of these files.
I didn't run the install procedure as I was working on my laptop with a dev
setup, just set the symlinks. Now fixed things as proposed by wajasu
on comment #4. Any other suggestion please let me know.
Tested to work on an up-to-date Ubuntu 12.04.
Asked by wajasu, remove remaining obsolete zebraqueue stuff.
Sponsored-by: Universidad Nacional de Córdoba
Signed-off-by: wajasu <matted-34813@mypacks.net>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Passed-QA-by: Mason James <mtj@kohaaloha.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
On some Linux distributions like RedHat, Fedora, CentOS you can use SELinux for enhanced security. Among others, this involves file labeling (security context). In other distributions SELinux can be installed additionally.
The attached script lets you update and restore such labels on the perl scripts in a Koha installation.
July 18, 2012: Added opac/svc.
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Prior patches to this bug had lots of comments like "I don't have a way to test this, so..."
In the OCLC Connexion web, when you choose the option to export to MARC, it'll *send* it, and
say, "Record Exported," but the web client does nothing whatever to confirm that the record
actually landed in Koha. That's a flaw in their software, but can be easily checked by
looking in Koha to see if an import batch got created. The desktop client is a little
smarter about this, but needed much more testing, also.
With this patch, both the client and web will actually work. With a config file and set up as
previously described, The record will be staged and/or imported, and the desktop client returns
a useful message about what happened, *and* the staff client URL to the record.
Oodles of gobs of bunches of thanks to Virginia Military Institute, for loaning me their OCLC
authorization credentials so this could be tested, as well as for great suggestions of cosmetic
improvements to the mechanism and output.
SIPServer.pm requires that C4/SIP is added to its lib
path This has been done by passing this directory
to it via -I. By using FindBin it can set the path
for itself correctly. This will also work if the C4/SIP
directory tree is moved to a non-standard location
Removed the now redundant -I. from sip_run.sh
Added a variable to sip_run.sh for the koha tree to
highlight a problem with the script if you have multiple
directories in the PERL5LIB environment variable
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
svc/import_bib:
* takes POST request with parameters in url and MARC XML as DATA
* pushes MARC XML to an impoort bach queue of type 'webservice'
* returns status and imported record XML
* is a drop-in replacement for svc/new_bib
misc/cronjobs/import_webservice_batch.pl:
* a cron job for processing impoort bach queues of type 'webservice'
* batches can also be processed through the UI
misc/bin/connexion_import_daemon.pl:
* a daemon that listens for OCLC Connexion requests and is compliant
with OCLC Gateway spec
* takes request with MARC XML
* takes import batch params from a config file and forwards the lot to
svc/import_bib
* returns status
ImportBatches:
* Added new import batch type of 'webservice'
* Changed interface to AddImportBatch() - now it takes a hashref
* Replaced batch_type = 'batch' with
batch_type IN ( 'batch', 'webservice' ) in some SELECTs
Signed-off-by: MJ Ray <mjr@phonecoop.coop>
Signed-off-by: Liz Rea <wizzyrea@gmail.com>
The scripts run with the caveat that you must specify the path to SIPconfig.xml. The followup previously attached should take care of that issue.
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
[1] Increase sleep interval between checks of zebraqueue
from 0.01 seconds to 0.50.
[2] Batch up commits of changes to the zebraqueue table
[3] If the same record appears multiple times in the queue,
handle only once.
[4] Properly postpone failures to process record deletes to
avoid spinning.
[5] Correct how queue entries are marked done - avoid skipping
an authority record update, e.g., if it has the same
ID number as a bib that was updated.
[6] Added a FIXME about a possible later enhancement to
batch up updates so that Zebra isn't told to commit
after each record.
No documentation changes.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
If running Zebra, try to locate zebrasrv and zebraidx
so that koha-zebra-ctl.sh can point to a functioning
zebrasrv.
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
The daemon control scripts (koha-zebra-ctl.sh, koha-zebraqueue-ctl.sh,
and koha-pazpar2-ctl.sh) are now copied and installed in a
runnable fashion for a 'dev'-mode install. By default
they are installed in the bin subdirectory of the runtime
directory.
Also:
* the control scripts now work if the EUID is other
than root (as would be expected for a 'dev' or 'single'
install).
* Split the SCRIPT_DIR installation target into
SCRIPT_DIR (scripts to copy regardless of install mode)
and SCRIPT_NONDEV_DIR (scripts to copy to SCRIPT_DIR
unless the install mode is 'dev').
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
* adding zebra daemons for managing server and queue processes
* improvements to the README.debian file
* Fixes to Search.pm since last series of commits broke zebra-based
searching (again)
* moving some files to new misc/bin and misc/cronjobs
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
