What we currently have:
Koha/ElasticSearch.pm
Koha/ElasticSearch/Indexer.pm
Koha/SearchEngine/Elasticsearch/QueryBuilder.pm
Koha/SearchEngine/Elasticsearch/Search.pm
What we want:
Koha/SearchEngine/Elasticsearch.pm
Koha/SearchEngine/Elasticsearch/Indexer.pm
Koha/SearchEngine/Elasticsearch/QueryBuilder.pm
Koha/SearchEngine/Elasticsearch/Search.pm
Test plan:
% git grep -i Koha::ElasticSearch
% git grep ElasticSearch|grep -v Catmandu::Store::ElasticSearch
should not return any result
Do a full reindex and search for records
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Rather than add/remove regular expressions to skip modules (like bug 9054), encapsulate the decision logic into a separate function.
Currently there are three libraries which trigger halts:
Koha::NorwegianDB (which was already there)
Koha::ElasticSearch::Indexer
Koha::SearchEngine::Elasticsearch::Search
TEST PLAN
---------
1) prove t/00-load.t
-- should barf horribly on Catmandu stuff
if not: sudo apt-get remove libcatmandu-marc-perl
then repeat step.
2) apply patch
3) prove t/00-load.t
-- should not barf horribly
4) run koha qa test tools
NOTE: The four optional modules for Koha::NorwegianDB are
listed in the PerlDependencies.pm, while there is
no mention of Catmandu libraries at all there.
This may be another bug which needs fixing.
TECH NOTES (for ideas of how to tinker around):
These three things should trigger the three module cases:
sudo apt-get remove libcatmandu-marc-perl
sudo apt-get remove libcatmandu-store-elasticsearch-perl
sudo apt-get remove libconvert-basen-perl
You probably had koha-perldeps installed before, so the following wil mostly fix:
sudo apt-get install koha-perldeps libcatmandu-marc-perl
And in case you didn't have elastic search stuff installed:
echo deb http://packages.elastic.co/elasticsearch/1.7/debian stable main | sudo tee /etc/apt/sources.list.d/elasticsearch.list
wget -O- https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
sudo apt-get update
cd ~
wget http://debian.koha-community.org/koha/otherthings/elasticsearch_deps.tar.gz
tar xvf elasticsearch_deps.tar.gz
cd es_deps
sudo dpkg i lib*
sudo apt-get install -f
Signed-off-by: Srdjan <srdjan@catalyst.net.nz>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Galen Charlton <gmcharlt@gmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
For historical reasons the SIPServer and SIP modules
have used an extra module path in addition to the
standard Koha one. This has caused numerous irritants
in attempting to set up scripts and basic tests. It
does not help in attempting to modify or debug
this code
This patch changes the package value in the modules
under the C4/SIP directory and makes calls to
them use the full package name.
Where the export mechanism was being short circuited
routines have been explicitly exported and imported
declarations of 'use ILS' when that module was
not being used and which only generated warnings
have been removed.
As a lot of the changes affect lines where
an object is instantiated with new. The opportunity
has been taken to replace the ambiguous indirect
syntax with the preferred direct call
In intializing ILS the full path is added as this
will not require any changes to existing configs.
I suspect this feature is unused, and adds
obfuscation rather than flexibility but have kept
the feature as we need this change in order to
rationalize and extend the testing of the server.
The visible difference is that with the normal Koha
PERL5LIB setting. Compilation of Modules under C4/SIP
should be successful and not fail with unlocated modules,
allowing developers to see any perl warnings
All the SIP modules can now be run through the tests
in t/00-load.t now except for SIPServer itself
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This patch suggests to create a routine to mock C4::Context::_new_dbh.
NOTE: Works the same with and without this secondary patch.
koha-qa tests fine. Less cutting and pasting in the future.
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
To reproduce:
- Stop your MySQL server:
$ sudo service mysql stop
- Run
$ prove t/00-load.t
=> FAIL: some tests fail because of mysql stopped
To test (MySQL still stopped)
- Apply the patch
- Run
$ prove t/00-load.t
=> SUCCESS: tests pass because the ycan be loaded regardless of
the absence of the DB server
- Sign off :-D
NOTE: Even seems to grab more than expected, which is good.
349 tests in master vs 364 in this branch = 16,
but removed block is only 13 (lines 20-32).
Also ran koha-qa test tool. :)
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Works as described, tests passing now without database available.
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This patch makes it possible to sync patron data between Koha and the
Norwegian national patron database, in both directions.
In order to use this, the following information is necessary:
- a username/password from the Norwegian national database of libraries
("Base Bibliotek"), available to all Norwegian libraries
- a special key in order to decrypt and encrypt PIN-codes/passwords,
which is only available to Norwegian library system vendors
- a norwegian library vendor username/password
See http://www.lanekortet.no/ for more information (in Norwegian).
While this is of course an implementation of a specific synchronization scheme
for borrower data, attempts have been made to prepare the ground for other sync
schemes that might be implemented later. Especially the structure of the new
borrower_sync table might be reviewed with an eye to how it might fit other
schemes.
To test:
Since the password and cryptographic key needed to use this functionality
is only available to Norwegian library system vendors, only regression testing
can be done on the submitted code. Suggested things to check:
- Apply the patch and make sure the database update is done. This should add
the new "borrower_sync" table and five new systmpreferences under the
"Patrons" > "Norwegian patron database" category:
- NorwegianPatronDBEnable
- NorwegianPatronDBEndpoint
- NorwegianPatronDBUsername
- NorwegianPatronDBPassword
- NorwegianPatronDBSearchNLAfterLocalHit
- Check that patrons can be created, edited and deleted as usual, when
NorwegianPatronDBEnable is set to "Disable"
- Check that the new tests in t/NorwegianPatronDB.pm run ok, e.g. on a
gitified setup:
$ sudo koha-shell -c "PERL5LIB=/path/to/kohaclone prove -v t/NorwegianPatronDB.t" instancename
- Check that all the other tests still run ok
- Check that the POD in the new files itroduced by this patch looks ok:
- Koha/NorwegianPatronDB.pm
- members/nl-search.pl
- misc/cronjobs/nl-sync-from-koha.pl
- misc/cronjobs/nl-sync-to-koha.pl
- t/NorwegianPatronDB.t
Sponsored-by: Oslo Public Library
Update 2014-09-18:
- Rebase on master
- Split out changes to Koha::Schema
- Incorporate new way of authenticating with NL
Update 2014-10-21:
- Rebase on master
- Use Module::Load to load Koha::NorwegianPatronDB in non-NL-specific
scripts and modules
- Fix the version number of Digest::SHA
- Fix a missing semicolon in kohastructure.sql
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Since nobody is currently working on the zebra layer introduced by bug
8233, Solr won't never work.
Some code has been introduced in 3.10 to prove several search engines
can cohabit into Koha but no help/fund has been found to go ahead.
It is useless to keep this code and to maintain an ambiguous situation.
I think the indexes configuration page could be restore later if someone
else introduces a new search engine into Koha.
Test plan:
Look at the code introduced by bug 8233 and verify all is removed.
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
To test:
- Apply first 2 patches for unit tests and changes to Services.pm
- Turn off mysql
- Run t/00-load.t
- Verify that the test fails.
- Apply this third patch.
- Run t/00-load.t again.
- Verify this time it passes.
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Comment: Work well, no koha-qa errors.
Without third patch and mysql down, test fails,
with third patch it succeed.
Signed-off-by: Mason James <mtj@kohaaloha.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
At the moment we cache numerous pieces of information in module-level
variables which then do not get updated in other threads/processes when
they are changed by the user. This is a serious usability issue.
Examples of this include the way we treat sysprefs (there is now a
method to disable the syspref cache, but by default it is enabled),
notices, frameworks, field mappings, and koha-conf.xml, at least.
This patch sets the stage for eliminating this problem by making it
possible to convert module-level cache variables into variables that
are actually backed by whatever caching system may be configured. This
is done through a special Koha::Cache::Object class which can be tied
to the variables that are being used for caching and provided with a
constructor method/closure to allow the cache to be reloaded when it
expires. For example:
my $cache = Koha::Cache->new();
my $data = 'whatever';
my $variable = Koha::Cache->create_scalar(
{
'key' => 'whatever',
'timeout' => 2,
'constructor' => sub { return $data; },
}
);
print "$$variable\n"; # Prints "whatever"
The one change this necessitates for accessing the data is that the
variable must be dereferenced an additional time before use (i.e.
$$variable instead of $variable). There is no difference when the
variable tied is a hash (created with Koha::Cache->create_hash). This
is a small price to pay for Koha working in a multi-threaded, persistent
environment. This change will also make caching easier in general.
CHI was incompatible with the variable tying, so this patch also removes
the dependency on CHI, using instead Cache::Memcached::Fast,
Cache::FastMmap, and Cache::Memory, when they are available.
To test:
1) Apply patch.
2) Run unit test t/Cache.t (after setting the MEMCACHED_SERVERS and
CACHING_SYSTEM environment variables). As no changes were made to
the tests already in that file, this passing demonstrates there
are no regressions.
3) With memcached caching enabled (you must set the MEMCACHED_SERVERS
and CACHING_SYSTEM environment variables) and DEBUG turned on (i.e.
the DEBUG environment variable set to 1), try running a report via
the web service ([intranet]/cgi-bin/koha/svc/report?id=1 and check
your web server logs to confirm that there are messages like
"get_from_cache for intranet:report:id:1" in them.
4) If the reports worked, sign off.
NOTE: Technically you could test this without needing memcached by
installing libcache-fastmmap-perl and setting CACHING_SYSTEM to
'fastmmap' instead of 'memcached'. You could also install libcache-perl
and set CACHING_SYSTEM to 'memory' but there would be little point as
the cached variables would go out of scope in between runs.
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Several patterns were being skipped unnecessarily in t/00-load.t:
* C4::Cache* - no longer exists
* C4::Record - no longer requires database
* C4::Serials - no longer requires database
To test:
1) Unset KOHA_CONF and/or shut down MySQL.
2) Run `prove t/00-load.t`
3) If the test passes, the patch is good.
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
144 tests passing.
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Prior to this patch, t/00-load.t was trying to load Koha::SearchEngine,
which fails when Moose is not installed.
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
C4::Members::Statistics uses the database in its BEGIN {} block,
which breaks the package build process. This patch causes it to be
skipped in the 00-load test.
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
This moves the DB-requiring tests out of the way, with the exception of
00-load.t which is used by the git hooks. For it, it makes it skip
loading problematic modules. This allows 'make test' to complete successfully
without a database configured, wich is a required part of making packages.
This has been tested against the v3.02.03 tag and the master branch.
Signed-off-by: Chris Nighswonger <chris.nighswonger@gmail.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
File::Spec is a core module, Path::Class isn't; not yet
worth adding a new dependency.
Also some minor whitepace cleanup.
Signed-off-by: Galen Charlton <gmcharlt@gmail.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Script to test modules compile, when used with a pre-commit hook this
can test before a commit
Signed-off-by: Frédéric Demians <f.demians@tamil.fr>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
