At the moment we cache numerous pieces of information in module-level
variables which then do not get updated in other threads/processes when
they are changed by the user. This is a serious usability issue.
Examples of this include the way we treat sysprefs (there is now a
method to disable the syspref cache, but by default it is enabled),
notices, frameworks, field mappings, and koha-conf.xml, at least.
This patch sets the stage for eliminating this problem by making it
possible to convert module-level cache variables into variables that
are actually backed by whatever caching system may be configured. This
is done through a special Koha::Cache::Object class which can be tied
to the variables that are being used for caching and provided with a
constructor method/closure to allow the cache to be reloaded when it
expires. For example:
my $cache = Koha::Cache->new();
my $data = 'whatever';
my $variable = Koha::Cache->create_scalar(
{
'key' => 'whatever',
'timeout' => 2,
'constructor' => sub { return $data; },
}
);
print "$$variable\n"; # Prints "whatever"
The one change this necessitates for accessing the data is that the
variable must be dereferenced an additional time before use (i.e.
$$variable instead of $variable). There is no difference when the
variable tied is a hash (created with Koha::Cache->create_hash). This
is a small price to pay for Koha working in a multi-threaded, persistent
environment. This change will also make caching easier in general.
CHI was incompatible with the variable tying, so this patch also removes
the dependency on CHI, using instead Cache::Memcached::Fast,
Cache::FastMmap, and Cache::Memory, when they are available.
To test:
1) Apply patch.
2) Run unit test t/Cache.t (after setting the MEMCACHED_SERVERS and
CACHING_SYSTEM environment variables). As no changes were made to
the tests already in that file, this passing demonstrates there
are no regressions.
3) With memcached caching enabled (you must set the MEMCACHED_SERVERS
and CACHING_SYSTEM environment variables) and DEBUG turned on (i.e.
the DEBUG environment variable set to 1), try running a report via
the web service ([intranet]/cgi-bin/koha/svc/report?id=1 and check
your web server logs to confirm that there are messages like
"get_from_cache for intranet:report:id:1" in them.
4) If the reports worked, sign off.
NOTE: Technically you could test this without needing memcached by
installing libcache-fastmmap-perl and setting CACHING_SYSTEM to
'fastmmap' instead of 'memcached'. You could also install libcache-perl
and set CACHING_SYSTEM to 'memory' but there would be little point as
the cached variables would go out of scope in between runs.
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
* Add "Plugins disabled" screen instead of error
* Allow plugins to return a value, add a test run that checks the return value
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Tests Ok
~/kohaclone$ perl t/db_dependent/Plugins.t
1..15
ok 1 - use Koha::Plugins;
ok 2 - use Koha::Plugins::Handler;
ok 3 - use Koha::Plugins::Base;
ok 4 - use Koha::Plugin::Test;
ok 5 - Test can_load
ok 6 - Test plugin class isa Koha::Plugin::Test
ok 7 - Test plugin parent class isa Koha::Plugins::Base
ok 8 - Test plugin can report
ok 9 - Test plugin can tool
ok 10 - Test plugin can configure
ok 11 - Test plugin can install
ok 12 - Test plugin can install
ok 13 - Test $plugin->get_metadata()
ok 14 - Test $plugin->get_qualified_table_name()
ok 15 - Test $plugin->get_plugin_http_path()
(and all others as well)
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
This patch removes 'insecure' system preference.
Also removes remaining code that make use of
the preference. It's broken anyway.
Only remains a reference in POD of C4/Boolean.pm
To test:
1) If you like, enable 'insecure' syspref. Broken system.
WARN: be prepared to revert value in database.
2) Apply the patch
3) Run updatedatabase.pl
4) Check that Staff login proceeds as usual.
5) Check that 'insecure' syspref is no more.
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Amended patch: Remove 2 occurrences of insecure (in comment only)
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Removed NoZebra vestiges. This comprises several code blocks that depend on the NoZebra syspref and NZ related functions/methods.
C4::Biblio->
GetNoZebraIndexes
_DelBiblioNoZebra
_AddBiblioNoZebra
C4::Search->
NZgetRecords
NZanalyse
NZoperatorAND
NZoperatorOR
NZoperatorNOT
NZorder
C4::Installer->
set_indexing_engine
Sponsored-by: Universidad Nacional de Córdoba
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
With the inclusion of this patch, all searches will (try) to use
QueryParser for handling queries for both the bibliographic and authority
databases if UseQueryParser is enabled. If QueryParser is unavailable,
UseQueryParser is disabled, or the search uses CCL indexes, the old
search code will be used.
To test:
1) Apply patch.
2) Run the unit test with `prove t/QueryParser.t`
3) Enable the UseQueryParser syspref.
4) Try searches that should return results in the following places:
* OPAC (simple search)
* OPAC (advanced search)
* OPAC (authorities)
* Staff client (header search)
* Staff client (advanced search)
* Staff client (cataloging search)
* Staff client (authorities)
* Staff client (importing a batch using a match point)
* Staff client (searching for an item for adding to a label)
* Staff client (acquisitions)
* Staff client (searching for a record to create a serial)
* ANYWHERE ELSE I HAVE FORGOTTEN
5) Disable the UseQueryParser syspref. Repeat at least some of the
searches you did above.
6) If all searches worked, sign off.
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Elliott Davis <elliott@bywatersolions.com>
Searching still works as expected for variuos places.
QueryParser syspref seemed to be enabled by default
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Since the most expressive query language supported by Zebra is PQF, this
patch adds a PQF driver for QueryParser which will translate QueryParser
queries into standard PQF (guided by mappings which have been written to
match Koha's existing Zebra configuration) which can then be sent to
Zebra. This driver, Koha::QueryParser::Driver::PQF(::*) extends the
OpenILS::QueryParser(::*) class(es), so as to preserve maximum
interoperability between the various users of the QueryParser driver.
Initially, search syntax is as follows:
* AND operator: &&
* OR operator: ||
* GROUPING operators: ( )
Fields can mostly be searched using the ccl prefixes they have now. The
exception is the various date limits which are searched with a syntax
like this: pubdate(2008)
For sorting, you can simply add #title-sort-az (etc.) to your query.
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Elliott Davis <elliott@bywatersolions.com>
Test Passed successfully after installing missing dep for Test::Deep
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
jcamins confirmed on IRC it is not optional.
To test:
1) perl koha_perl_deps.pl -a |grep String::Random
2) notice the "No" in the last column - it should be Yes
3) apply patch and repeat same steps, looking for Yes.
4) prove t/Installer_PerlModules.t # which checks String::Random is required
Sponsored-by: software.coop
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Prior to this patch, C4::VirtualShelves::Page did not make sure that
the itemtype images that it chose when displaying itemtype images for
biblio-level itemtypes were for the correct interface, so
even on the OPAC the Intranet icons were requested. On standard
installations, intranet-tmpl is not available to the OPAC.
To test:
1) Set noItemTypeImages to "Show" and item-level_items to "biblio record"
2) Create a list and add an item with the default item type (942$c in
MARC21) set to something with an itemtype image associated with it.
3) View the list in the OPAC.
4) If you are an a standard install, the itemtype image will be a broken
link. If on a dev install, you can confirm with Firebug or the like
that the image is pulled from /intranet-tmpl.
5) Apply patch.
6) Run `prove t/db_dependent/Koha.t` to confirm that the behavior of the
routine used to retrieve the image URL is correct. (Note that you
must have an item type with the code 'BK')
7) Refresh the list, noting that the display is now correct/the image is
pulled from opac-tmpl.
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
It works as expected. All tests in Koha.t passed.
Signed-off-by: Mason James <mtj@kohaaloha.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Because the unit test t/Letters.t loads C4::Letters before C4::Context,
and C4::Letters is not even vaguely thread-safe, the test tends to fail.
Usually. Moving the dbh mocking to before the use_ok('C4::Letters') test
fixes the problem.
To test:
1) Before applying patch, run `prove t/Letters.t` a few times. Note that
it fails most of the time, if not all the time.
2) Apply patch.
3) Repeat step (1), noting that now it passes every time.
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as advertised.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Passed-QA-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Added RentalsInNoissueCharges and ManInvlsInNoissueCharges sys prefs
Created C4::Members::cwGetMemberAccountBallance()
* A wrapper for GetMemberAccountRecords that gives info on non-issue and
other charges
* Other charges are:
'Res'
'Rent' if RentalsInNoissueCharges is Mo
authorised_values MANUAL_INV if ManInvlsInNoissueCharges is No
C4::Members::GetMemberAccountRecords() changes:
* Dropped input param $date, it is not used
Use split charges in C4::Circulation::CanBookBeIssued() and
C4::Members::patronflags(). That way only fines decide whether an item
can be issued, and not other non-fine charges
Signed-off-by: Marc Veron <veron@veron.ch>
Rebased (updatedatabase.pl)
ManInvInNoissueCharges and RentalsInNoissueCharges ar both included by default (= behaviour as before)
All variants tested: Both included, none included, manual invoice included, rentals included.
Works fine, blocks/does not blok as appropirate, messages appear as expected.
[Oct 12, 2012 marcelr:] Amended for updatedatabase.pl
Signed-off-by: M. de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Several patterns were being skipped unnecessarily in t/00-load.t:
* C4::Cache* - no longer exists
* C4::Record - no longer requires database
* C4::Serials - no longer requires database
To test:
1) Unset KOHA_CONF and/or shut down MySQL.
2) Run `prove t/00-load.t`
3) If the test passes, the patch is good.
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
144 tests passing.
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Because the calendar tests require a more recent version of DBD::Mock
than has been packaged by Debian, it makes sense to skip all the tests
requiring the database when only an older version is present.
Signed-off-by: Liz Rea <liz@catalyst.net.nz>
Works as advertised
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Using specific method for populating the internal data structures from Koha::Calendar
has yielded to the non-detection of several bugs. There are also several tests that where
db_dependent which is not always desirable.
I propose the use of DBD::Mock (::Session) for using the actual code used by Koha in production
for testing, mocking the DB queries itselves.
I also took the time to repeat several tests in different syspref configurations (they applied
only to daysMode=Calendar, and now cover all confs).
Notes:
- I used DBD:Mock 1.45 as previous version (1.43, from 12.04) was broken
- Some tests revealed a bug on days_between as I see it... reporting as Bug #9211
Sponsored-by: Universidad Nacional de Córdoba
Signed-off-by: Elliott Davis <elliott@bywatersolutions.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Adjusts calling conventions to use hashrefs and eliminate redundant
procedural/OO mixed code.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
The patch for bug 9191 included example updatedatabase stanzas that
require removal before pushing. This patch also updates the
00-checkdatabase-version.t test so that it will not give
false-positives.
Check whether a given update should be run when passed the proposed
version number. The update will always be run if the proposed version
is greater than the current database version and less than or equal to
the version returned by C4::Context->final_linear_version (initially set
to be equal to the version in kohaversion.pl). The update is also run if
the version contains XXX, though this behavior will be changed following
the adoption of non-linear updates as implemented in bug 7167.
To test:
1) Make sure that the first example database update added by this patch
in installer/data/mysql/updatedatabase.pl has a version number one
greater than the version of Koha you have installed.
2) Set the Version syspref back one version behind your current version.
3) Navigate to the main page of the staff client, and log in to the
installer.
4) Confirm that the update page claims to have rerun the previous update
and has displayed the log message:
"Upgrade to 3.11.00.XXX done (Bug 9191: You should see this)"
but not the log message:
"Upgrade to [version number] done (Bug 9191: You shouldn't see this)"
Note: the sample database revisions will be removed by the RM before
this patch is pushed to master.
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
works great
Signed-off-by: Elliott Davis <elliott@test.bywatersolutions.com>
Works as expected. Test plan is great. Code looks to be adherent to standards.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
- Reintroduce the class column in a select statement (was present before
this development)
- FIX UT: If there are no arg passed in parameter, the routine calls
C4::Context->userenv which call an execute statement. It consumes the
mocked resultset
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Elliott Davis <elliott@test.bywatersolutions.com>
All tests successful.
Files=1, Tests=9, 0 wallclock secs ( 0.02 usr 0.01 sys + 0.16 cusr 0.06 csys = 0.25 CPU)
Result: PASS
[This patch was split out from tcohen's excellent patches for bug 8519
--jcamins 2012/10/31]
Added a new koha-index-daemon-ctl.sh script that uses the
'koha-index-daemon' provided by Koha::Contrib::Tamil (install via CPAN)
to process the indexing queue ('zebraqueue' for now, 'indexqueue' in the
future I guess :-P).
This script could be easily modified in the future for using another
index queue processing script (a more abstract indexing script for both Solr
and Zebra, etc). I removed all zebra-ish stuff from it.
Documentation patches will be provided as followups, once Mark Tompsett's
docs are pushed to avoid continous rebasing.
Tested to work on an up-to-date Ubuntu 12.04.
Sponsored-by: Universidad Nacional de Córdoba
Signed-off-by: wajasu <matted-34813@mypacks.net>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Debian stable's version of Test::More is older than the one I used on my
12.04 dev box, and doesn't support subtests, which I used to avoid side
effects between different test scenarios.
This patch removes that subtest definition.
Signed-off-by: Robin Sheat <robin@catalyst.net.nz>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
All tests pass.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
useDaysMode=Datedue wasn't used as advertised in the docs. Added
next_open_day and prev_open_day subs to Koha::Calendar and some tests for them.
- Koha::Calendar->addDate was rewritten in a more sane way (also split into
addHours and addDays for convenience).
- Fixed a bug introduced in Bug 8966 regarding dt truncation and dtSets->contains
- Minor docs typos
- Use the passed Calendar mode or default to 'Calendar' in Koha::Calendar->_mockinit.
- Tests
I'm writing some db-dependent tests for is_holiday, and hopefully for CalcDateDue so any
rewrite/followup doesn't break things.
Regards
To+
Sponsored-by: Universidad Nacional de Córdoba
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passed-QA-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Prior to this patch, t/00-load.t was trying to load Koha::SearchEngine,
which fails when Moose is not installed.
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Because C4::Context uses an in-memory hash for caching sysprefs,
changing a syspref under a multi-threaded persistent environment
requires a server restart. This patch makes it possible disable
the syspref cache.
To test:
1) If you are using a multi-threaded persistent server (Starman, etc.),
change a syspref and note that the effects of the syspref change may
or may not be visible on any given request before applying this patch.
You will need to choose a syspref with obvious effects that can be
seen by simply refreshing the page. I recommend enabling or
disabling additional languages in the OPAC, since you can refresh the
page a dozen times and reasonably expect to see the new behavior you
set only 1/n of the time.
2) Apply patch.
3) Add "C4::Context->disable_syspref_cache();" to your koha.psgi file
4) Repeat step 1, noting that you never see the stale behavior.
5) Run test at t/db_dependent/Context.t.
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
All tests passed.
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
[This patch was split out from tcohen's excellent patches for bug 8519
--jcamins]
It removes the obsolete zebraqueue_daemon.pl and koha-zebraqueue-ctl.sh
obsolete scripts too.
Several files are modified to address te removal/addition of these files.
I didn't run the install procedure as I was working on my laptop with a dev
setup, just set the symlinks. Now fixed things as proposed by wajasu
on comment #4. Any other suggestion please let me know.
Tested to work on an up-to-date Ubuntu 12.04.
Asked by wajasu, remove remaining obsolete zebraqueue stuff.
Sponsored-by: Universidad Nacional de Córdoba
Signed-off-by: wajasu <matted-34813@mypacks.net>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Passed-QA-by: Mason James <mtj@kohaaloha.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
The test was moved since the original patch was written, making the
revised test fail. This patch corrects the mocked template patch so that
the test will pass.
To test:
1) Before applying patch, run
`prove t/db_dependent/SuggestionEngine_ExplodedTerms.t`
2) Note that it fails.
3) Apply patch.
4) Run `prove t/db_dependent/SuggestionEngine_ExplodedTerms.t`
5) Note that it passes.
6) Celebrate.
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Even though there is no need for anything stored in the database for the
test, C4::Templates requires the database and a koha-conf.xml. The
solution is to mock all database- and koha-conf-using routines.
To test:
1) Stop MySQL
2) Unset KOHA_CONF
3) Run test
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
* Create a local copy of the date instead of calling truncate directly on the date
* Add a test to properly test that the issue is fixed
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
C4::Members::Statistics uses the database in its BEGIN {} block,
which breaks the package build process. This patch causes it to be
skipped in the 00-load test.
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Works better now, creats biblio records, and cleans up after itself as
well.
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
This fixes the display of different sized words for differently popular
tags. It is a bit of a refactor of that part of the system, moving
logic to more sensible places (and removing an unused method on the
way.)
Note that it isn't an attempt to reproduce what was there previously,
just to do something similar, and in an easier to change fashion.
Sponsored-By: New Zealand Educational Institute
Signed-off-by: Marc Veron <veron@veron.ch>
Works as expected.
This test file is meant to be used with the sample database
with a minimum of the sample libraries and sample holidays installed.
http://bugs.koha-community.org/show_bug.cgi?id=8110
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Fixes several perlcritic complaints, and some overflow in the
configuration page. Also notes that Did You Mean is not yet
enabled on the staff client.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Sorry about the problematic last attempt at uploading. I missed a
patch when squashing. It is now included.
When working with hierarchical subject headings, it is sometimes helpful
to do a search for all records with a specific subject, plus
broader/narrower/related subjects. This patch adds a suggestion plugin for
these "exploded" subject searches to Koha. Note that this patch depends on
both bug 8211 AND bug 8209.
To test (NOTE: this test plan covers both 8211 and 8726):
1) Make sure you have a bunch of hierarchical subjects. I created
geographical subjects for "Arizona," "United States," and "Phoenix,"
and linked them together using 551s, and made sure I had a half
dozen records linking to each (but not all to all three).
2) Do a search for su-br:Arizona (or choose "Subject and broader terms"
on the advanced search screen with "more options" displayed), and
check that you get the records with the subject "Arizona" and the
records with the subject "United States"
3) Do a search for su-na:Arizona (or choose "Subject and narrower terms"
on the advanced search screen with "more options" displayed), and
check that you get the records with the subject "Arizona" and the
records with the subject "Phoenix"
4) Do a search for su-rl:Arizona (or choose "Subject and related terms"
on the advanced search screen with "more options" displayed), and
check that you get the records with the subject "Arizona," the
records with the subject "United States," and the records with the
subject "Phoenix"
5) Ensure that other searches still work (keyword, subject, ccl,
whatever)
6) Use "Did you mean?" page in admin section to enable ExplodedTerms
plugin
7) Do a keyword search on the OPAC, confirm that searching for exploded
terms is suggested.
8) Do a subject search on the OPAC, confirm that searching for exploded
terms is suggested.
9) Do a non-keyword, non-subject search on the OPAC, confirm that
searching for exploded terms is NOT suggested.
10) Disable ExplodedTerms plugin and enable AuthorityFile plugin.
11) Do search on OPAC, confirm suggestions are made from authority file.
12) Sign off
Signed-off-by: wajasu <matted-34813@mypacks.net>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Split into two patches. This patch includes only the functionality.
This commit adds support for displaying authority hierarchies for all
flavours of MARC, not just UNIMARC. Display now uses the jQuery
jstree plugin, selected with the help of Owen Leonard, resulting in a
much faster experience for users.
Be aware that the jstree file uses tabs rather than 4-space indentation,
which I left as-is so as to make it easier to integrate upstream
releases in the future.
To test:
1) Enable the AuthDisplayHierarchy syspref
2) Create authority records with a hierarchy of see also fields
(in MARC21/NORMARC, you'll be using 5xx fields for this, with a
subfield $w=g for broader terms and subfield $w=h for narrower
terms)
3) View the authorities in the OPAC, noting the hierarchical view at
the top of the page.
This initial patch does not create bidirection linkages from
unidirectional links in MARC21 authorities. This means that when moving
up the authority hierarchy, lower levels will disappear. This is
intentional, as the first patch is intended merely to ensure that
AuthDisplayHierarchy functions the same for all marcflavours. A future
patch will add a cron job to generate the bidirectional linkages, once
we are sure that the hierarchy functionality for UNIMARC and
MARC21/NORMARC coexists peaceably.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Resolved conflicts in updatedatabase.pl, sysprefs.sql and in one of
the CSS files.
Test plan:
1) Run t/AuthoritiesMarc.t
New tests complete without any errors.
2) Make sure updatedatabase works correctly.
Update works nicely, new system preference is also added to syspref.sql
3) Make sure new terms are translatable.
Created new po files for de-DE and checked for new terms.
All translations appear correctly.
4) Make sure everything works with AuthDisplayHieararchy OFF
- Add authority
- Edit authority
- Delete authority
5) Test feature with AuthDisplayHieararchy ON
- Add authority
- Edit authority
- Delete authority
6) Add a couple of hierarchically linked authorities
Note: links have to be created in both directions
Example:
151 $aGermany
551 $a Baden-Württemberg $w h
151 $aBaden-Württemberg
551 $a Konstanz $w h
551 $a Germany $w g
151 $aKonstanz
551 $a Baden-Württemberg $w g
551 $a Fürstenberg $w h
551 $a Paradies $w h
151 $a Fürstenberg
551 $a Konstanz $w g
151 $a Paradies
551 $a Konstanz $w g
Tree shows up nicely above the authority record
- in staff
- in OPAC
- on the normal view tab
- on the MARC view tab
7) Checking the logs for warnings
- no Javascript errors or warnings
- no warnings or errors in log files
Signed-off-by: Pierre Angot <tredok.pierre@gmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Larry Baerveldt <larry@bywatersolutions.com>
Signed-off-by: Joy Nelson <joy@bywatersolutions.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
These tests use DBD::Mock to check if SQL queries are correctly built.
Actually, we only check bound parameters.
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
When working with hierarchical subject headings, it is sometimes helpful
to do a search for all records with a specific subject, plus
broader/narrower/related subjects. This patch adds support for these
"exploded" subject searches to Koha.
To test:
1) Make sure you have a bunch of hierarchical subjects. I created
geographical subjects for "Arizona," "United States," and "Phoenix,"
and linked them together using 551s, and made sure I had a half
dozen records linking to each (but not all to all three).
2) Do a search for su-br:Arizona (or choose "Subject and broader terms"
on the advanced search screen with "more options" displayed), and
check that you get the records with the subject "Arizona" and the
records with the subject "United States"
3) Do a search for su-na:Arizona (or choose "Subject and narrower terms"
on the advanced search screen with "more options" displayed), and
check that you get the records with the subject "Arizona" and the
records with the subject "Phoenix"
4) Do a search for su-rl:Arizona (or choose "Subject and related terms"
on the advanced search screen with "more options" displayed), and
check that you get the records with the subject "Arizona," the
records with the subject "United States," and the records with the
subject "Phoenix"
5) Ensure that other searches still work (keyword, subject, ccl,
whatever)
6) Sign off
Technical details:
This patch adds a shim in front of C4::Search::buildQuery in order to
preprocess the query and call the _handle_exploding_search callback.
This shim will allow us to gradually offload query parsing to a new
query parser module.
Signed-off-by: wajasu <matted-34813@mypacks.net>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
The second test in ItemType.t tests that an empty table
returns no rows however as Mock::DBD has no resultset
it fails with an error in DBI It requires that an
empty resultset is defined in Mock::DBD first
This patch adds that mocked empty table
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
This patch adds the Koha::Indexer::RecordNormalizer and
Koha::Indexer::MARC::RecordNormalizer::EmbedSeeFromHeadings packages
to enable the inclusion of alternate forms of headings in bibliographic
searches. When the new syspref IncludeSeeFromInSearches is turned on
(default is off) rebuild_zebra.pl will insert see from headings from
authority records into bibliographic records when indexing, so that a
search on an obsolete term will turn up relevant records.
To test:
1) Enable IncludeSeeFromInSearches
2) Add a heading that has an alternate form to a record (for example,
"Cooking" has the alternate form "Cookery," if you have authority
records from LC)
3) Index the zebraqueue (or reindex if you haven't indexed your system
yet)
4) Confirm that if you search for "Cookery" you get the record you
just modified
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on master 5 August 2012
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on master 11 September 2012
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Also checked:
- Verified database update works correctly
- Checked system preference and its description
- Checked staff/opac detail pages with feature on/off
- Checked staff/opac search facets
- Downloaded and tested records in various formats
- Tried different searches for 'see from' entries of authorities
- Ran all unit tests
No problems found.
The Zebra test uses Unix-domain sockets, which won't work if
Koha is being installed or tested from an NFS- or SSHFS-mounted
clone. Therefore, create (and clenaup) a temporary directory on
(hopefully) a normal filesytem to run the test.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: wajasu <matted-34813@mypacks.net>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This fixes failures of the test cases to run for me
on Perl 5.10.1 using prove -v t/db_dependent/Search.t
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: wajasu <matted-34813@mypacks.net>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This patch adds a unit test for C4::Search in t/db_dependent. In order
to test the functioning of the Zebra search, this patch actually includes
an entire Zebra sandbox, and pre-indexed files, which are stored in
t/db_dependent/data (the configuration files are generated on the fly).
This test depends on Test::Warn, Test::MockModule, and DBD::Mock.
To test:
1) Run the test. There should be no failures, and no warnings.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: wajasu <matted-34813@mypacks.net>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
One of the ideas behind authority records is that users who search for
one term should have related terms (according to the authority file)
suggested to them. At the moment, Koha doesn't do that. Adding an
authority searching step to regular searches and displaying any
suggestions in a "Did you mean" bar at the top of the results would be
very useful.
This commit adds a Koha::SuggestionEngine class which is in charge of
getting suggestions from individual suggestion engine plugins, which
much be in the Koha::SuggestionEngine::Plugin::* namespace, and extend
Koha::SuggestionEngine::Base. Suggestions are loaded asynchronously
using AJAX, and a link to a page with suggestions is provided for users
with Javascript turned off.
The AuthorityFile suggestion engine plugin looks up the specified search
terms in the authority file and then suggests searches for records using
matching authorities.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased 2 August 2012 and incorporated QA feedback
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Test plan:
- Verified database update added system preference correctly,
pref defaults to OFF
- Verified search results and detail pages in OPAC and staff
still worked the same as before
* for no results
* with results
- Activated system preference and tested various searches
* Searches from simple search
* Searches from advanced search
* Search links in records
- Deactivated Javascript - verified fallback works correctly
Notes:
- Suggested terms can include autorities with no linked records.
- When combining more than one search option using advanced search
this results in "no suggestions" more often. Feature works best
from simple search.
Overall great feature making use of authorities in a user friendly way!
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on latest master 2012-09-10
Signed-off-by: wajasu <matted-34813@mypacks.net>
Create transport_cost table, added UseTransportCostMatrix syspref.
transport_cost table contains branch to branch transfer
costs. These are used for filling inter-branch hold transfers.
Moved GetHoldsQueueItems() from .pl to HoldsQueue.pm
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
This reverts commit 215abc8024.
The 3 patches for bug 8089 have been reverted, because they break
jenkins & Koha.
A follow-up has been provided, but it does not solve the problem on my
test server, it just changes the error message.
After a discussion with jared, Dobrica should work on another patch, so
the best option is to revert.
1. Replace all instances of memoize_memcached with appropriate calls
into Koha::Cache:
* reports/guided_reports.pl
* C4::Biblio::GetMarcStructure
* C4::Languages::getFrameworkLanguages
* C4::Languages::getAllLanguages
* C4::SQLHelper::GetPrimaryKeys
* C4::SQLHelper::_get_columns
2. Replace all references to memcached with the appropriate calls into
Koha::Cache in C4::Context.
Test plan :
* have DEBUG env set to 1
* reach addbiblio page to test the patch in Biblio.pm, or setup more than 1
language
* you should see in the logs that you're reading and writing from cache
* run the test suite twice both with and without the following environment
variables set:
export MEMCACHED_SERVERS=127.0.0.1:11211
export MEMCACHED_NAMESPACE=KOHA
export CACHING_SYSTEM=memcached
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
I'm unsure about some of the caching times 10000 is a long long time,
but other than that, works fine.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
tested the 9 previous patches, that introduces many new unit tests using
the Mock Module.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
If you had a report query that had a subquery in the fields list, and
that subquery had a LIMIT specifier, then it would be removed which
could break your query. This patch prevents this case from breaking by
ensuring that only a LIMIT that follows the last WHERE in the query is
removed.
If you don't have a WHERE, then it will behave like it always
did, removing all the cases of LIMIT (which would still break a subquery
but this is a) more rare, and b) would require more intelligent parsing
to deal with.
Also adds test cases and function documentation.
Signed-off-by: Nicole C. Engard <nengard@bywatersolutions.com>
Tested with this report:
select biblionumber, (select itemnumber from items where items.biblionumber=biblio.biblionumber LIMIT 1) from biblio where biblionumber<1000;
and it worked like a charm
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
The t/Barcodes_ValueBuilder.t test had the wrong number of tests
declared, and rather than checking that a variable was undefined I was
checking that it had a length of zero. Fixed both issues.
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
In addition to adding a new barcode plugin, this commit begins
refactoring the barcode generation code using a new module,
C4::Barcodes::ValueBuilder. From the POD:
This module is intended as a shim to ease the eventual transition
from having all barcode-related code in the value builder plugin
barcode.pl file to using C4::Barcodes. Since the shift will require
a rather significant amount of refactoring, this module will return
value builder-formatted results, at first by merely running the
code that was formerly in the barcodes.pl value builder, but later
by using C4::Barcodes.
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Implement auto-incrementing EAN-13 barcodes
To make this work, C4::Barcodes::next was modified to call process_tail with
new incremented value so that process_tail can generate correct checksum.
Since process_tail is currenlty not used by any barcodes, this change is safe.
C4::Barcodes is used by addbiblio.pl when adding multiple records, while value_builder
is used in all other cases.
Test scenario:
1. prove t/Barcodes_EAN13.t
2. KOHA_CONF=/etc/koha/sites/fer/koha-conf.xml prove t/db_dependent/Barcodes.t
this will check C4::Barcode implementataion
3. in systempreference change autoBarcode to incremental EAN-13 barcode
4. edit two items of any biblio assigning barcodes and verify that numbers
are increasing. Have in mind that last digit is check digit, and it
doesn't increment, but is calculated from barcode itself. Example with
checksum in brackets: 000000086275[2], 000000086276[9], 000000086277[6]
5. Add Item and verify that it gets assigned next barcode
6. Add & Duplicate item and verify barcode increase
7. Add Multiple Copies and verify that barcode increase for each copy
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
This filter which check validity of EAN-13 barcode and padd it with zeros
up to full 13 digit number. This will also expand 12 digit UPC-A barcodes
to EAN-13 automatically which is useful for older barcode readers which tend
to ignore first zero in EAN-13 if they have just UPC-A support.
It should be noted that EAN-13 or UPC-A product codes printed on books are not
good choice for barcodes in Koha since each item has to have unique barcode.
Test scenario:
1. prove t/Circulation_barcodedecode.t
this checks expansion of 12 digit UPC-A to 13 digit EAN-13 and zero padding
2. in systempreferences search for itemBarcodeInputFilter and select EAN-13
3. edit one item and assign it valid EAN-13 barcode, eg. 0000000695152, check it out
4. test checkin with just 695152 to test leading zero expansion
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Makes the test db independant by specifying the date format. Thus, there is no call to ->preference() that retrieve a systempreference
Moves the tests previously in t/Kalendar.t to t/Calendar.t
and removes the old file.
Note the existing minimal test of C4/Calendar still called in
t/Calendar
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Thee were errors in the calculation used for days_between
which caused incorrect values to be returned
Added tests to validate calculation
NB Tests still need to be provided for the hourly
calculations
Adds a non-test mode (--showsql option) that displays the missing
sysprefs as sql inserts. Some sysprefs were missed from the syspref
file due to extra spaces in the insert : fixed. The test now checks if
there are at least as many sysprefs in the database as in the syspref
file (instead of checking for the exact number)
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Cherrypicked from BibLibre master, rebased onto current master, and
corrected typos, 2012-07-03
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
This patch adds t/db_dependent/00-strict.pl which tests that all Perl
scripts compile. I chose to use Test::Strict so that in the future we
can also have the test check that "use strict" and "use warnings" are
enabled.
To test:
1. Run t/db_dependent/00-strict.pl.
2. Run around waving your arms in the air that we have a couple of
scripts that don't compile.
3. Actually, that's about it. You could test my patch for bug 8384 next.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Moved test to t/db_dependent
Signed-off-by: Frédéric Demians <f.demians@tamil.fr>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
modified: Koha.t
$ prove -v ./Koha.t
1..5
ok 1 - use C4::Koha;
ok 2 - use C4::Members;
1..6
ok 1 - Insert data in database
ok 2 - GetAuthorisedValueByCode
ok 3 - GetKohaImageurlFromAuthorisedValues
ok 4 - lost and paid works
ok 5 - Child area works
ok 6 - Withdrawn works
ok 3 - Authorized Values Tests
1..4
ok 1 - C4::Koha->can('GetDailyQuote')
ok 2 - Got a quote based on id.
ok 3 - Got a random quote.
ok 4 - Got a quote based on today's date.
ok 4 - Daily Quotes Test
1..7
ok 1 - slashifyDate
ok 2 - xml_escape() returns empty string on undef input
ok 3 - xml_escape() works as expected
ok 4 - ... and does not change input in place
ok 5 - _isbn_cleanup removes hyphens
ok 6 - _isbn_cleanup removes parenthetical
ok 7 - _isbn_cleanup converts ISBN-13 to ISBN-10
ok 5 - Date and ISBN tests
ok
All tests successful.
Files=1, Tests=5, 1 wallclock secs ( 0.02 usr 0.01 sys + 0.48 cusr 0.06 csys = 0.57 CPU)
Result: PASS
http://bugs.koha-community.org/show_bug.cgi?id=5327
Add C4/SIP to perlcritic tests. Fix those issues that were
generating perlcritic errors
Signed-off-by: Stacey Walker <stacey@catalyst-eu.net>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
- in various acquisition pages and serials home
- in database : biblioitems.ean
- adds ean and its mapping in default english bibliographic framework
- adds ean mapping in default french bibliographic framework
- ean search is not enabled for MARC21
The required mapping between the ean marc field and the biblioitems.ean
database field will be automatically added on an existing unimarc installation.
However, if you already have records with ean, you will have to
run misc/batchRebuildBiblioTables.pl to populate biblioitems.ean
Signed-off-by: jmbroust <jean-manuel.broust@univ-lyon2.fr>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Passed QA at second run. Removed a merge marker only.
Implements cache handlers for Memcached, mmap shared-file persistent,
and in-process memory caches. If CHI is unavailable, Koha::Cache::Memcached
will fall back to using Cache::Memcached::Fast, or caching will be skipped
without croaking.
To test: run t/Cache.t with the following options, before and after
installing CHI:
2. Tests 3-9 should be skipped with the following:
export CACHING_SYSTEM=
2. You should receive two failures with the following if CHI is not installed:
export CACHING_SYSTEM=memory
3. You should receive two failures with the following if CHI is not installed:
export CACHING_SYSTEM=fastmmap
4. You will need memcached activated for the following to work (but it
will work both with and without CHI):
export CACHING_SYSTEM=memcached
export MEMCACHED_SERVERS=127.0.0.1:11211
export MEMCACHED_NAMESPACE=KOHA
5. You should receive two failures with the following:
export CACHING_SYSTEM=thisdoesntexist
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Works as advertised, now we need a follow up to add the new dependency.
http://bugs.koha-community.org/show_bug.cgi?id=8029
Signed-off-by: Nicole C. Engard <nengard@bywatersolutions.com>
preference works as expected.
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Tested that CircAutoPrintQuickSlip clear the screen still works well
Unit test for Koha/Cache.pm (which includes Koha/Cache/Memcached.pm)
Note that in order to test Koha::Cache you must export the environment variable
MEMCACHED_SERVERS. For example:
$ export MEMCACHED_SERVERS=127.0.0.1:11211
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
If there are no late orders in the database, the 5th unit test for C4/Acquisition.pm
fails, since no date is returned (and today's date is expected)
This patch adds a conditional check around whether any late orders are actually returned
by GetLateOrders. If so, tests proceed, if not, they're skipped. This seems to quiet
the failures in the automated test suite.
http://bugs.koha-community.org/show_bug.cgi?id=5347
signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
patch applied to commit eb3dc448d2
Signed-off-by: Nicole C. Engard <nengard@bywatersolutions.com>
Turned on star ratings in the opac on details and results
Searched for titles - saw the stars
Clicked on a title
Clicked on the stars
Clicked on the stars to change my rating
Logged out
Tried to click on stars
Logged in as different user
Rated items that were rated already and saw average change
Changed preference to show only on detail and repeated tests
Changed preference to now show stars
All above tests passed. Signing off.
Rebased 3-19-12 by Ian Walls
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
svc/import_bib:
* takes POST request with parameters in url and MARC XML as DATA
* pushes MARC XML to an impoort bach queue of type 'webservice'
* returns status and imported record XML
* is a drop-in replacement for svc/new_bib
misc/cronjobs/import_webservice_batch.pl:
* a cron job for processing impoort bach queues of type 'webservice'
* batches can also be processed through the UI
misc/bin/connexion_import_daemon.pl:
* a daemon that listens for OCLC Connexion requests and is compliant
with OCLC Gateway spec
* takes request with MARC XML
* takes import batch params from a config file and forwards the lot to
svc/import_bib
* returns status
ImportBatches:
* Added new import batch type of 'webservice'
* Changed interface to AddImportBatch() - now it takes a hashref
* Replaced batch_type = 'batch' with
batch_type IN ( 'batch', 'webservice' ) in some SELECTs
Signed-off-by: MJ Ray <mjr@phonecoop.coop>
Skip the MARC21-specific tests in t/Heading_MARC21.t if the marcflavour is
UNIMARC.
Signed-off-by: Magnus Enger <magnus@enger.priv.no>
I set marcflavour = UNIMARC for testing this.
Before the patch, t/Heading_MARC21.t fails. After the patch it
succeeds.
Signed-off-by: Magnus Enger <magnus@enger.priv.no>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
One of the tests in t/db_dependent/Linker_FirstMatch.t ony works if Zebra is
running and authorities have been indexed. This was causing Jenkins agitas.
The test should be skipped if Zebra isn't running or authorities have not been
indexed at the time the test is run.
Signed-off-by: Ian Walls <koha.sekjal@gmail.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
If the book is due at 23:59 the due time is irrelevant
and potentially confusing to the user add a DateUtils
routine that strips that off the returned display date
Add testscript for DateUtils
Add a parameter to override system pref to output_pref not for
use in calling software (its superfluous) but to enable testing
of call with different settings
Branches can have their own version of notices - added branchcode to
letter table.
Support html notices - added is_html to letter table.
Support for borrower attributes in templates.
GetPreparedletter() is the interface for compiling letters (notices).
Sysprefs for notice and slips stylesheets
Added TRANSFERSLIP to the letters
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Squashed patch incorporating all previous patches (there is no functional
change compared to the previous version of this patch, this patch merely
squashes the original patch and follow-up, and rebases on latest master).
=== TL;DR VERSION ===
*** Installation ***
1. Run installer/data/mysql/atomicupdate/bug_7284_authority_linking_pt1
and installer/data/mysql/atomicupdate/bug_7284_authority_linking_pt2
2. Make sure you copy the following files from kohaclone to koha-dev:
etc/zeradb/authorities/etc/bib1.att,
etc/zebradb/marc_defs/marc21/authorities/authority-koha-indexdefs.xml,
etc/zebradb/marc_defs/marc21/authorities/authority-zebra-indexdefs.xsl,
etc/zebradb/marc_defs/marc21/authorities/koha-indexdefs-to-zebra.xsl, and
etc/zebradb/marc_defs/unimarc/authorities/record.abs
3. Run misc/migration_tools/rebuild_zebra.pl -a -r
*** New sysprefs ***
* AutoCreateAuthorities
* CatalogModuleRelink
* LinkerModule
* LinkerOptions
* LinkerRelink
* LinkerKeepStale
*** Important notes ***
You must have rebuild_zebra processing the zebraqueue for bibs when testing this
patch.
=== DESCRIPTION ===
*** Cataloging module ***
* Added an additional box to the authority finder plugin for "Heading match,"
which consults not just the main entry but also See-from and See-also-from
headings.
* With this patch, the automatic authority linking will actually work properly
in the cataloging module. As Owen pointed out while testing the patch,
though, longtime users of Koha will not be expecting that. In keeping with
the principles of least surprise and maximum configurability, a new syspref,
CatalogModuleRelink makes it possible to disable authority relinking in the
cataloging module only (i.e. leaving it enabled for future runs of
link_bibs_to_authorities.pl). Note that though the default behavior matches
the current behavior of Koha, it does not match the intended behavior.
Libraries that want the intended behavior rather than the current behavior
will need to adjust the CatalogModuleRelink syspref.
*** misc/link_bibs_to_authorities.pl ***
Added the following options to the misc/link_bibs_to_authorities.pl script:
--auth-limit Only process those headings that match the authorities
matching the user-specified WHERE clause.
--bib-limit Only process those bib records that match the
user-specified WHERE clause.
--commit Commit the results to the database after every N records
are processed.
--link-report Display a report of all the headings that were processed.
Converted misc/link_bibs_to_authorities.pl to use POD.
Added a detailed report of headings that linked, did not link, and linked
in a "fuzzy" fashion (the exact semantics of fuzzy are up to the individual
linker modules) during the run.
*** C4::Linker ***
Implemented new C4::Linker functionality to make it possible to easily add
custom authority linker algorithms. Currently available linker options are:
* Default: retains the current behavior of only creating links when there is
an exact match to one and only one authority record; if the 'broader_headings'
option is enabled, it will try to link to headings to authority records for
broader headings by removing subfields from the end of the heading (NOTE:
test the results before enabling broader_headings in a production system
because its usefulness is very much dependent on individual sites' authority
files)
* First Match: based on Default, creates a link to the *first* authority
record that matches a given heading, even if there is more than one
authority record that matches
* Last Match: based on Default, creates a link to the *last* authority
record that matches a given heading, even if there is more than one record
that matches
The API for linker modules is very simple. All modules should implement the
following two functions:
<get_link ($field)> - return the authid for the authority that should be
linked to the provided MARC::Field object, and a boolean to indicate whether
the match is "fuzzy" (the semantics of "fuzzy" are up to the individual plugin).
In order to handle authority limits, get_link should always end with:
return $self->SUPER::_handle_auth_limit($authid), $fuzzy;
<flip_heading ($field)> - return a MARC::Field object with the heading flipped
to the preferred form. At present this routine is not used, and can be a stub.
Made the linking functionality use the SearchAuthorities in C4::AuthoritiesMarc
rather than SimpleSearch in C4::Search. Once C4::Search has been refactored,
SearchAuthorities should be rewritten to simply call into C4::Search. However,
at this time C4::Search cannot handle authority searching. Also fixed numerous
performance issues in SearchAuthorities and the Linker script:
* Correctly destroy ZOOM recordsets in SearchAuthorities when finished. If left
undestroyed, efficiency appears to approach O(log n^n)
* Add an optional $skipmetadata flag to SearchAuthorities that can be used to
avoid additional calls into Zebra when all that is wanted are authority
records and not statistics about their use
*** New sysprefs ***
* AutoCreateAuthorities - When this and BiblioAddsAuthorities are both turned
on, automatically create authority records for headings that don't have
any authority link when cataloging. When BiblioAddsAuthorities is on and
AutoCreateAuthorities is turned off, do not automatically generate authority
records, but allow the user to enter headings that don't match an existing
authority. When BiblioAddsAuthorities is off, this has no effect.
* CatalogModuleRelink - when turned on, the automatic linker will relink
headings when a record is saved in the cataloging module when LinkerRelink
is turned on, even if the headings were manually linked to a different
authority by the cataloger. When turned off (the default), the automatic
linker will not relink any headings that have already been linked when a
record is saved.
* LinkerModule - Chooses which linker module to use for matching headings
(current options are as described above in the section on linker options:
"Default," "FirstMatch," and "LastMatch")
* LinkerOptions - A pipe-separated list of options to set for the authority
linker (at the moment, the only option available is "broader_headings," which
is described below)
* LinkerRelink - When turned on, the linker will confirm the links for headings
that have previously been linked to an authority record when it runs. When
turned off, any heading with an existing link will be ignored.
* LinkerKeepStale - When turned on, the linker will never *delete* a link to an
authority record, though, depending on the value of LinkerRelink, it may
change the link.
*** Other changes ***
* Cleaned up authorities code by removing unused functions and adding
unimplemented functions and added some unit tests.
* This patch also modifies the authority indexing to remove trailing punctuation
from Match indexes.
* Replace the old BiblioAddAuthorities subroutines with calls into the new
C4::Linker routines.
* Add a simple implementation for C4::Heading::UNIMARC. (With thanks to F.
Demians, 2011.01.09) Correct C4::Heading::UNIMARC class loading. Create
biblio tag to authority types data structure at initialization rather than
querying DB.
* Ran perltidy on all changed code.
*** Linker Options ***
Enter "broader_headings" in LinkerOptions. With this option, the linker will
try to match the following heading as follows:
=600 10$aCamins-Esakov, Jared$xCoin collections$vCatalogs$vEarly works to
1800.
First: Camins-Esakov, Jared--Coin collections--Catalogs--Early works to 1800
Next: Camins-Esakov, Jared--Coin collections--Catalogs
Next: Camins-Esakov, Jared--Coin collections
Next: Camins-Esakov, Jared (matches! if a previous attempt had matched, it
would not have tried this)
This is probably relevant only to MARC21 and LCSH, but could potentially be of
great use to libraries that make heavy use of floating subdivisions.
=== TESTING PLAN ===
Note: all of these tests require that you have some authority records,
preferably for headings that actually appear in your bibliographic data. At
least one authority record must contain a "see from" reference (remember which
one contains this, as you'll need it for some of the tests). The number shown
in the "Used in" column in the authority module is populated using Zebra
searches of the bibliographic database, so you *must* have
rebuild_zebra.pl -b -z [-x] running in cron, or manually run it after running
the linker.
*** Testing the Heading match in the cataloging plugin ***
1. Create a new record, and open the cataloging plugin for an
authority-controlled field.
2. Search for an authority by entering the "see from" term in the Heading Match
box
3. Confirm that the appropriate heading shows up
4. Search for an authority by entering the preferred heading into the Main
entry or Main entry ($a only) box (i.e., repeat the procedure you usually
use for cataloging, whatever that may be)
5. Confirm that the appropriate heading shows up
*** Testing the cataloging interface ***
6. Turn off BiblioAddsAuthorities
7. Confirm that you cannot enter text directly in an authority-controlled field
8. Confirm that if you search for a heading using the authority control plugin
the heading is inserted (note, however, that this patch does not AND IS NOT
INTENDED TO fix the bugs in the authority plugin with duplicate subfields;
those are wholly out of scope- this check is for regressions)
9. Turn on BiblioAddsAuthorities and AutoCreateAuthorities
10. Confirm that you can enter text directly into an authority-controlled field,
and if you enter a heading that doesn't currently have an authority record,
an authority record stub is automatically created, and the heading you
entered linked
11. Confirm that if you enter a heading with only a subfield $a that fully
*matches* an existing heading (i.e. the existing heading has only
subfield $a populated), the authid for that heading is inserted into
subfield $9
12. Confirm that if you enter a heading with multiple subfields that *matches*
an existing heading, the authid for that heading is inserted into
subfield $9
13. Turn on BiblioAddsAuthorities and turn off AutoCreateAuthorities
14. Confirm that you can enter text directly into an authority-controlled field,
and if you enter a heading that doesn't currently have an authority record,
an authority record stub is *not* created
15. Confirm that if you enter a heading with only a subfield $a that *matches*
an existing heading, the authid for that heading is inserted into
subfield $9
16. Confirm that if you enter a heading with multiple subfields that *matches*
an existing heading, the authid for that heading is inserted into
subfield $9
17. Create a record and link an authority record to an authorized field using
the authority plugin.
18. Save the record. Ensure that the heading is linked to the appropriate
authority.
19. Open the record. Change the heading manually to something else, leaving
the link. Save the record.
20. Ensure that the heading remains linked to that same authority.
21. Change CatalogModuleRelink to "on."
22. Open the record. Use the authority plugin to link that heading to the
same authority record you did earlier.
23. Save the record. Ensure that the heading is linked to the appropriate
authority.
24. Open the record. Change the heading manually to something else, leaving
the link. Save the record.
25. Ensure that the heading is no longer linked to the old authority record.
*** Testing link_bibs_to_authorities.pl ***
26. Set LinkerModule to "Default," turn on LinkerRelink and
BiblioAddsAuthorities, and turn AutoCreateAuthorities and
LinkerKeepStale off
27. Edit one bib record so that an authority controlled field that has already
been linked (i.e. has data in $9) has a heading that does not match any
authority record in your database
28. Run misc/link_bibs_to_authorities.pl --link-report --verbose --test (you may
want to pipe the output into less or a file, as the result is quite a lot of
information)
29. Look over the report to see if the headings that you have authority records
for report being matched, that the heading you modified in step 2 is
reported as "unlinked," and confirm that no changes were actually made to
the database (to check this, look at the bib record you edited earlier, and
check that the authid in the field you edited hasn't changed)
30. Run misc/link_bibs_to_authorities.pl --link-report --verbose (you may want
to pipe the output into less or a file, as the result is quite a lot of
information)
31. Check that the heading you modified has been unlinked
32. Change the modified heading back to whatever it was, but don't use the
authority control plugin to populate $9
33. Run misc/link_bibs_to_authorities.pl --link-report --verbose
--bib-limit="biblionumber=${BIB}" (replacing ${BIB} with the biblionumber
of the record you've been editing)
34. Confirm that the heading has been linked to the correct authority record
35. Turn LinkerKeepStale on
36. Change that heading to something else
37. Run misc/link_bibs_to_authorities.pl --link-report --verbose
--bib-limit="biblionumber=${BIB}" (replacing ${BIB} with the biblionumber
of the record you've been editing)
38. Confirm that the $9 has not changed
39. Turn LinkerKeepStale off
40. Create two authorities with the same heading
41. Run misc/migration_tools/rebuild_zebra.pl -a -z
42. Enter that heading into the bibliographic record you are working with
43. Run misc/link_bibs_to_authorities.pl --link-report --verbose
--bib-limit="biblionumber=${BIB}" (replacing ${BIB} with the biblionumber
of the record you've been editing)
44. Confirm that the heading has not been linked
45. Change LinkerModule to "FirstMatch"
46. Run misc/link_bibs_to_authorities.pl --link-report --verbose
--bib-limit="biblionumber=${BIB}" (replacing ${BIB} with the biblionumber
of the record you've been editing)
47. Confirm that the heading has been linked to the first authority record it
matches
48. Change LinkerModule to "LastMatch"
49. Run misc/link_bibs_to_authorities.pl --link-report --verbose
--bib-limit="biblionumber=${BIB}" (replacing ${BIB} with the biblionumber
of the record you've been editing)
50. Confirm that the heading has been linked to the second authority record it
matches
51. Run misc/link_bibs_to_authorities.pl --link-report --verbose
--auth-limit="authid=${AUTH}" (replacing ${AUTH} with an authid)
52. Confirm that only that heading is displayed in the report, and only those
bibs with that heading have been changed
If all those things worked, good news! You're ready to sign off on the patch
for bug 7284.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on latest master and squashed follow-up, 16 February 2012
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on latest master, 21 February 2012
Signed-off-by: schuster <dschust1@gmail.com>
Adds a new placeholder {ISSN} to the system preference SearchForTitleIn.
For a record with multiple ISSNs only the first ISSN will be used.
Addition: Makes a small change to GetMarcControlnumber so that it checks for
NORMARC too. If you set your system preference to NORMARC, it should output
{CONTROLNUMBER} correctly now.
For testing add following code to the system preference and check output
of SearchForTitleIn for different records in your OPAC and all 3 available
views (normal, MARC and ISBD):
<li>ISSN: {ISSN}</li>
<li>ISBN: {ISBN}</li>
<li>001: {CONTROLNUMBER}</li>
Patch also includes some unit tests:
perl t/db_dependent/Biblio.t
Signed-off-by: Magnus Enger <magnus@enger.priv.no>
Tested with marcflavour = NORMARC, on one book and one periodical record.
* Book
- Before the patch:
ISSN: {ISSN}
ISBN: 0375726446
001:
- After the patch:
ISSN:
ISBN: 0375726446
001: 022976914
* Journal
- Before the patch:
ISSN: {ISSN}
ISBN:
001:
- After the patch:
ISSN: 1890-6931
ISBN:
001: 080721370
Looks good in all 3 views! Thanks for fixing the 001 thing for NORMARC!
Also tested with marcflavour = MARC21, on the same records with the same good
results. Signing off!
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Tested marcflavour= UNIMARC, works fine too
C4::Biblio is used in many many places. The goal of this cleaning is to do from C4::Biblio a package with as many dependancies as possible.
* C4::Heading is called only in 1 place, highly rarely used (only in 1 misc/link_bibs_to_authorities.pl), moving to require
* PrepareItemrecordDisplay is a sub that is more related to Items, moving it here. It means some scripts that used this sub must be checked against use C4::Items
* C4::Items is needed in EmbedItemsInMarcBiblio, moving it only in this sub, and switching to require
* 2 subs are totally useless z3950_extended_services and set_service_options, removing them
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
No test plan included, but tested some opac and cataloging functions.
Code looks good. Marked as Passed QA.
Display suggestion info in acquisition module:
basket.pl
neworderempty.pl
orderreceive.pl
parcel.pl
To Test:
Create a suggestion and accept it.
Create a new order from this suggestion
Receive this order
For each step, check if suggestion info are visible.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Test provides more unit tests, all complete successfully.
perl t/db_dependent/Suggestions.t
Note: test case should be cleaned up after running tests.
Display changes are consistent and information about the suggestion
is shown on every important screen now.
I created an order from a suggestion and one from an existing record.
No problems found.
