From the POD of Koha::Cache->set_in_cache:
# This is a bit of a hack to support the old API in case things still use it
Let's remove this hack and update old calls.
Test plan:
Look at the results of
git grep set_in_cache
and confirm that there are no more version of the old call (without
hashref as third param)
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
We need to define several namespaces for our cache system.
For instance sysprefs, koha conf (koha-conf.xml) and unit tests
should be defined in a separate namespace.
This will permit to
- launch the tests without interfering with other cache values
- and flush the sysprefs cache without flushing all other values
To do so, we need to store different Koha::Cache objects at a package
level. That's why this patch adds a new Koha::Caches module.
FIXME: There is an architecture problem here: the L1 cache should be
defined in Koha::Cache
Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
https://bugs.koha-community.org/show_bug.cgi?id=11921
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
https://bugs.koha-community.org/show_bug.cgi?id=11921
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Thou shalt not forget to include changes made in the t/Cache.t
in the patch set.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
The goal of this patch is to avoid unecessary flush of the L1 cache on
creating a new CGI object each time C4::Languages::getlanguage is called
without a CGI object.
The new class Koha::Cache::Memory::Lite must be flushed by the CGI
constructor overide done in the psgi file. This new class will ease
caching of specific stuffs used by running script.
Test plan:
At the OPAC and the intranet interfaces:
Open 2 different browser session to simulate several users
- Clear the cookies of the browsers
- User 1 (U1) an User 2 (U2) should be set to the default language
(depending on the browser settings)
- U1 chooses another language
- U2 refreshes and the language used must be the default one
- U2 chooses a third language
- U1 refreshes and must be still using the one he has choosen.
Try to use a language which is not defined:
Add &language=es-ES (if es-ES is not translated) to the url, you should
not see the Spanish interface.
Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Could be useful later.
Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>
Signed-off-by: Brendan Gallagher <bredan@bywatersolutions.com>
Koha::Cache->set_in_cache should deep copy (if needed) to avoid the
value which has been set in cache to be unintentionally modified later.
Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>
Signed-off-by: Brendan Gallagher <bredan@bywatersolutions.com>
When a value exists in L2 cache but not in L1 cache, it should be deep
copied if needed (i.e. not a scalar). Otherwise the calling code is able
to modify the value in cache.
Note that is theoretical, it's possible that no code does that.
Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>
Signed-off-by: Brendan Gallagher <bredan@bywatersolutions.com>
If the caller/developer knows what he is doing, he can decide not to
deep copy the structure. It will be faster but unsafe!
If the structure is modified, the cache will also be updated.
This option must be used with care and is not the default behavior.
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
The timeout does not impact the L1 cache (it would be to time consuming
and not really useful to do that for this cache).
To simulate the real timeout, we need to flush this L1 cache when
needed.
It would be also done adding a disable_L1_cache method.
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
If the user runs:
$ prove t/Cache.t
with it's system user, two situations can happen:
1) If MEMCACHED_NAMESPACE is set on koha-httpd.xml other than the default 'koha', then
Apache sets /tmp/sharefile-koha-<namespace> and the problem is not present: running
the test creates /tmp/sharefile-koha-koha != /tmp/sharefile-koha-<namespace>
=> SUCCESS: no problem
2) If MEMCACHED_NAMESPACE is not set (or eq 'koha'), then there is a permission problem
either running the unit tests, or when using any funcitonality on the UI that needs
Koha::Cache.
Explanation: the one that is run first will set the /tmp/sharefile-koha-koha ownership
so it will be either the dev's sys user, or www-data (or whatever apache is using).
This patch sets a namespace for the unit tests, so there is no collision.
To test:
- On your dev setup, having MEMCACHED_NAMESPACE unset on koha-httpd.conf
- Edit a marc framework. If it fails, remove /tmp/sharefile-koha-koha, and try again
-> fixed. Now try running
$ prove t/Cache.t
=> FAIL: test fails because of permission problem
- Apply the patch
- Re-run the test
=> SUCCESS: test passes
Try changing the order, etc.
The temporary file that is used is deleted after the tests are run.
Sponsored-by: Universidad Nacional de Cordoba
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This makes Koha::Cache behave better by default. It will use memcached
if available to do shared caching, if that's not available it will fall
back to in-memory caching. It also allows for a singleton accessor to
allow a single cache to be shared within a process.
* Added tests to confirm UTF8-cleanness.
* Added minor fixups to stop warnings.
Test plan:
* The t/Cache.t file runs successfully with and without the
MEMCACHED_SERVERS envvar set (and memcached running in the
environment.)
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
At the moment we cache numerous pieces of information in module-level
variables which then do not get updated in other threads/processes when
they are changed by the user. This is a serious usability issue.
Examples of this include the way we treat sysprefs (there is now a
method to disable the syspref cache, but by default it is enabled),
notices, frameworks, field mappings, and koha-conf.xml, at least.
This patch sets the stage for eliminating this problem by making it
possible to convert module-level cache variables into variables that
are actually backed by whatever caching system may be configured. This
is done through a special Koha::Cache::Object class which can be tied
to the variables that are being used for caching and provided with a
constructor method/closure to allow the cache to be reloaded when it
expires. For example:
my $cache = Koha::Cache->new();
my $data = 'whatever';
my $variable = Koha::Cache->create_scalar(
{
'key' => 'whatever',
'timeout' => 2,
'constructor' => sub { return $data; },
}
);
print "$$variable\n"; # Prints "whatever"
The one change this necessitates for accessing the data is that the
variable must be dereferenced an additional time before use (i.e.
$$variable instead of $variable). There is no difference when the
variable tied is a hash (created with Koha::Cache->create_hash). This
is a small price to pay for Koha working in a multi-threaded, persistent
environment. This change will also make caching easier in general.
CHI was incompatible with the variable tying, so this patch also removes
the dependency on CHI, using instead Cache::Memcached::Fast,
Cache::FastMmap, and Cache::Memory, when they are available.
To test:
1) Apply patch.
2) Run unit test t/Cache.t (after setting the MEMCACHED_SERVERS and
CACHING_SYSTEM environment variables). As no changes were made to
the tests already in that file, this passing demonstrates there
are no regressions.
3) With memcached caching enabled (you must set the MEMCACHED_SERVERS
and CACHING_SYSTEM environment variables) and DEBUG turned on (i.e.
the DEBUG environment variable set to 1), try running a report via
the web service ([intranet]/cgi-bin/koha/svc/report?id=1 and check
your web server logs to confirm that there are messages like
"get_from_cache for intranet:report:id:1" in them.
4) If the reports worked, sign off.
NOTE: Technically you could test this without needing memcached by
installing libcache-fastmmap-perl and setting CACHING_SYSTEM to
'fastmmap' instead of 'memcached'. You could also install libcache-perl
and set CACHING_SYSTEM to 'memory' but there would be little point as
the cached variables would go out of scope in between runs.
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
This reverts commit 215abc8024.
The 3 patches for bug 8089 have been reverted, because they break
jenkins & Koha.
A follow-up has been provided, but it does not solve the problem on my
test server, it just changes the error message.
After a discussion with jared, Dobrica should work on another patch, so
the best option is to revert.
1. Replace all instances of memoize_memcached with appropriate calls
into Koha::Cache:
* reports/guided_reports.pl
* C4::Biblio::GetMarcStructure
* C4::Languages::getFrameworkLanguages
* C4::Languages::getAllLanguages
* C4::SQLHelper::GetPrimaryKeys
* C4::SQLHelper::_get_columns
2. Replace all references to memcached with the appropriate calls into
Koha::Cache in C4::Context.
Test plan :
* have DEBUG env set to 1
* reach addbiblio page to test the patch in Biblio.pm, or setup more than 1
language
* you should see in the logs that you're reading and writing from cache
* run the test suite twice both with and without the following environment
variables set:
export MEMCACHED_SERVERS=127.0.0.1:11211
export MEMCACHED_NAMESPACE=KOHA
export CACHING_SYSTEM=memcached
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
I'm unsure about some of the caching times 10000 is a long long time,
but other than that, works fine.
Implements cache handlers for Memcached, mmap shared-file persistent,
and in-process memory caches. If CHI is unavailable, Koha::Cache::Memcached
will fall back to using Cache::Memcached::Fast, or caching will be skipped
without croaking.
To test: run t/Cache.t with the following options, before and after
installing CHI:
2. Tests 3-9 should be skipped with the following:
export CACHING_SYSTEM=
2. You should receive two failures with the following if CHI is not installed:
export CACHING_SYSTEM=memory
3. You should receive two failures with the following if CHI is not installed:
export CACHING_SYSTEM=fastmmap
4. You will need memcached activated for the following to work (but it
will work both with and without CHI):
export CACHING_SYSTEM=memcached
export MEMCACHED_SERVERS=127.0.0.1:11211
export MEMCACHED_NAMESPACE=KOHA
5. You should receive two failures with the following:
export CACHING_SYSTEM=thisdoesntexist
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Works as advertised, now we need a follow up to add the new dependency.
http://bugs.koha-community.org/show_bug.cgi?id=8029
Unit test for Koha/Cache.pm (which includes Koha/Cache/Memcached.pm)
Note that in order to test Koha::Cache you must export the environment variable
MEMCACHED_SERVERS. For example:
$ export MEMCACHED_SERVERS=127.0.0.1:11211
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
