Commit graph

181 commits

Author SHA1 Message Date
0a53d5e6b6 Bug 12651: DOM indexing is the default
On the 23 July development meeting it was decided to formally deprecate
GRS-1 indexing mode for Zebra. This patch makes code fallback to DOM
on the remaining places. No behaviour change should be noticed, as DOM
has been the default for a while.

Regards

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes tests and QA script.
Also checked running Makefile.PL

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2014-10-27 12:35:44 -03:00
41168fb551 Bug 12005 : Creating a new zebra connection for each time we need one
Zebra is not designed to have persistent connections, under cgi this
didn't matter the scripts would get a new connection each time, but
under plack we try to use dead connections

This patch changes it so plack works the same way that cgi did.

To test:
Apply this patch
Do some searches
Check everything still works

Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>

Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>

Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2014-10-22 14:29:23 -03:00
3c9004357d BUG8446, Follow up: Improve local login fallback
- Local fallback was not very well implemented, this patch adds
  better handling for such cases allowing clearer failure messages
- This patch also adds the ability to use single sign on via the
  top bar menu in the bootstrap theme.

BUG8446, Follow up: Adds perldoc documentation

- Add some documentation to the Auth_with_Shibboleth module
  including some guidance as to configuration.

BUG8446, Follow up: Correct filenames to match guidlines

- Moved Auth_with_Shibboleth.pm to Auth_with_shibboleth.pm to match
  other files present on the system.

BUG8446, Follow up: Correct paths after file rename

BUG8446, Follow up: Implemented single sign out

- This follow up rebases the code against 3.16+ which managed to break
  some of the original logic.
- As a side effect of the rebasing, we've also implemented the single
  sign out element. Upon logout, koha will request that the shibboleth
  session is destroyed, and then clear the local koha session upon
  return to koha.  Due to the nature of shibboleth however, you will
  only truly be signed out of the IdP if they properly support Single
  Sign Out (which many do not). As a consequence, although you may
  appear to be logged out in koha, you might find that upon clicking
  'login' the IdP does NOT request your login details again, but instead
  logs you silently back into your koha session. This is NOT a koha bug,
  but a shibboleth implementation issue that is well known.

BUG8446, Follow up: Fixed bootstrap login via modal

- The bootstrap theme enable login from any opac page via modal. To
  enable this with shibboleth we had to make some template parameters
  globally accessible when shibboleth is enabled.

BUG8446, Follow up: Add template rules for Shibboleth and CAS

- Add template rules so that CAS and Shibboleth can coexist.

BUG8446, Follow up: Added default config to config file

BUG8446, Follow up: Embellished perldoc documentation

- Updated perldoc to correct detail about configuring shibboleth
  authentication.
- Updated perldoc to include subroutines and their respective functions.

BUG8446, Follow up: Enable configuration of match field

- Added clearer, more flexible, configuration of shibboleth attribute to
  koha borrower field matching for authentication
- Correcting of documentation to make it more clear to the current
  implementation
- Minor refactoring of code to reduce some code duplication

Signed-off-by: Matthias Meusburger <matthias.meusburger@biblibre.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2014-10-16 12:27:51 -03:00
80a1225bb7 Bug 12738: C4::Context should set 'keyword' as the default search class
On a Perl 5.18 environment (e.g. Ubuntu 14.04 LTS):

To reproduce:
- Have UseQueryParser == "Don't try"
- Perform a search typing just a word that would have many matches on your Zebra db.
- Take note of the search results count.
- Make sure you have the queryparser.yaml file in your config dir.
- Set UseQueryParser == "Try"
- Perform the same search a couple of times
=> FAIL: Notice that the search results count differs from one run to the other.
- Run the regression tests
  $ prove -v t/db_dependent/QueryParser.t
=> FAIL: tests fail

To test:
- Apply the patch
- Run tests
  $ prove -v t/db_dependent/QueryParser.t
=> SUCCESS: tests pass
- Repeat the steps 'To reproduce'
=> SUCCESS: Search results count is deterministic :-D
- Sign off :-D

Sponsored-by: Universidad Nacional de Cordoba
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Now search with QueryParser always returns the correct number of
results on a system with the newer Perl version.
Passes all tests and QA script.

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2014-08-11 10:06:07 -03:00
fb2345a302 Bug 9612: (follow-up) restore elementSetName in Context.pm
Restore elementSetName to marcxml for DOM indexing in Zconn (Context.pm).
This prevents the need of rebuilding the index after restarting Zebra
server.

Removes the now incorrect reference to marcxml as 'superfluous' in four
dom config files.

Test plan:
[1] Do not yet apply this patch.
[2] Rebuild zebra index with the zebra config of commit
    036f2a50e1.
[3] (Go back to master.) Restart your zebra server (no config change).
    You will have results without details.
    Apply this patch: you see details.
    Reset to master: no details again.
[4] Install new zebra config from master.
    Search again: you still see no details.
    Restart zebra server. Search: you see details.
    Apply this patch. Search: still details.
    Restart zebra server. Search: still details.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Tested in a non-package environment (manual dev install).
The package environment should work now too (results in step 4c might differ).
Progress on bug 12012 would be appropriate to sync all changes.
Tested the response of the SRU server too.

Signed-off-by: Marc Veron <veron@veron.ch>
I tested starting on a VM with Koha 3.15.00.019 installed.
Did git pull -> Koha 3.15.00.051
Result: No details in search results.
Applied patch.
Result: Search results display fine.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-05-19 16:46:57 +00:00
e590436ee4 Bug 9612: Zconn parameters and elementSetName in _new_Zconn
This patch makes a few adjustments in C4::Context::Zconn.
It does no longer pass the (ignored) auth, syntax parameters to _new_Zconn.
Note that auth was not completely ignored in theory, but we never pass
auth=1 [while not having user/password in koha-conf].

Furthermore, it removes the elementSetName for dom indexing. Using marcxml
here does not make a difference. It only adds a constraint on what is in
the dom-config files. (It could probably be removed there now..)

Two cosmetic code changes:
Removes unused label retry.
And moved 'servername' into the database name option.

Test plan:
When using Zebra with dom indexing, do a biblio and authority search.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-05-05 20:31:16 +00:00
Julian Maurice
801c5e77d0 Bug 11848: Fix C4::Context::interface, add POD and UT
1/ Edit a Perl script, for example mainpage.pl
2/ add "use Koha::I18N;" to the top of file
3/ add a translatable message somewhere in the script (this have
   to be after the call to get_template_and_user). For example:
   warn gettext("This is a translated warning");
4/ Create or update the PO files with
   misc/translator/translate create LANGCODE
or
   misc/translator/translate update LANGCODE
   (LANGCODE should be enable in syspref 'languages')
5/ In misc/translator/po/LANGCODE-messages.po you should have
   your string, translate it (using a text editor or a PO file
   editor, make sure you don't have the "fuzzy" flag for this
   string).
6/ Go to mainpage.pl with active language being English with your
   browser and check your logs. You should see your string "This
   is a translated warning".
7/ Now change language to LANGCODE. Check your logs, you should
   have the string translated.

Note: I chose to name the sub 'gettext' because it's the default
keyword for xgettext for Perl. We can change it to whatever we want.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>

Follow test plan, work as described.
No koha-qa errors.
Tests pass

Fixed small merge conflict on t/Context.t

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.
Copied test plan from bug.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-05-05 04:32:31 +00:00
Julian Maurice
76e39750b7 Bug 11848: Move language detection function in C4::Languages
Also store interface (intranet, opac) in context to not have to pass it
as parameter.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
No koha-qa errors

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Comments on last patch.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-05-05 04:29:34 +00:00
Julian Maurice
9b63f0d018 Bug 11701: Fix C4::Context::Zconn
When calling C4::Context::Zconn twice with different parameters,
the same ZOOM::Connection object is returned twice (parameters
of 2nd call are not used) This patch fixes that.

This is in part because the connection cache is keyed on server
name only. This patch corrects this by keying on all parameters.

TEST PLAN
---------
1) apply patch
2) run koha qa test tools
3) prove -v t/Context.t

The unit tests properly triggers the modified routine for
testing. Additionally, in hunting for ways it could break,
no nested synchronous or asynchronous Zconn's were found.
And even if they were, the keying on all parameters should allow
it to function properly without messing up the other connection.

Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-03-25 14:43:03 +00:00
Galen Charlton
7d39b3c72e Bug 10611: (follow-up) cache DBMS driver code in context object
Rather than use a package-level variable, use an object-level
one.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-03-10 23:17:26 +00:00
Jonathan Druart
4ae3665ad8 Bug 10611: Use mysql_auto_reconnect instead of ping
DBD::Mysql provides a mysql_auto_reconnect flag. Using it avoids
the time required to do a $dbh->ping().

Benchmarks:

use Modern::Perl;
use C4::Context;
for ( 1 .. 1000 ) {
    $dbh = C4::Context->dbh;
}

* without this patch on a local DB:
perl t.pl  0,49s user 0,02s system 98% cpu 0,525 total
* without this patch on a remote DB:
perl t.pl  0,52s user 0,05s system 1% cpu 37,358 total
* with this patch on a local DB:
perl t.pl  0,46s user 0,04s system 99% cpu 0,509 total
* with this patch on a remote DB:
perl t.pl  0,49s user 0,02s system 56% cpu 0,892 total

Testing the auto reconnect:
use Modern::Perl;
use C4::Context;
my $ping = $dbh->ping;
say $ping;
$dbh->disconnect;
$ping = $dbh->ping;
say $ping;

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Comment: Real improvement. No koha-qa errors

prove t/db_dependent/Circulation_issuingrules.t produces no error
prove t/db_dependent/Context.t produces no error

Test
1) dumped Koha DB, load it on a non-local server
2) run sample script whit and without patch, local and remote

use Modern::Perl;
use C4::Context;
for ( 1 .. 100000 ) {
    my $dbh = C4::Context->dbh;
}

Main difference I note is with remote server
a) without patch
real    0m16.357s
user    0m2.592s
sys     0m2.132s

b) with patch
real    0m0.259s
user    0m0.240s
sys     0m0.012s

I think this could be good for DBs placed on
remote servers

Bug 10611: add a "new" parameter to C4::Context->dbh

When dbh->disconnect is called and the mysql_auto_reconnect flag is set,
the dbh is not recreated: the old one is used.

Adding a new flag, we can now force the C4::Context->dbh method to
return a new dbh.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>

Bug 10611: Followup: remove useless calls to dbh->disconnect

These 3 calls to disconnect are done at the end of the script, they are
useless.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-03-10 23:15:35 +00:00
Mark Tompsett
bf3b1aac7b Bug 11587: get rid of warnings generated by IsSuperLibrarian with anonymous sessions
This corrects line 1250 of C4/Context.pm to be:
    return ($userenv->{flags}//0) % 2;
And thus avoids an uninitialized value used in the modulus.

TEST PLAN
---------
1) Apply the first patch (to update t/Context.t)
2) prove -v t/Context.t
-- This should fail tests 7 and 8
3) Apply this patch (to fix C4/Context.pm)
4) prove -v t/Context.t
-- All tests should succeed

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-03-10 16:05:57 +00:00
daf2ebc4f5 Bug 11096: support the retrieval of large MARCXML records
This patch makes Koha <-> Zebra use MARCXML for the serialization when
using DOM, and USMARC for GRS-1.

* The following functions are modified to set the Zebra record syntax
according to the current sysprefs and configuration:

- C4::Context->Zconn
- C4::Context-_new_Zconn

* A new function 'new_record_from_zebra' is introduced, which checks the
context we are in, and creates the MARC::Record object using the right
constructor.

The following packages get touched to make use of the new function:
- C4::Search
- C4::AuthoritiesMarc

and the same happens to the UI scripts that make use of them (both in
the OPAC and STAFF interfaces).

* Calls to the unsafe ZOOM::Record->render()[1] method are removed.

Due to this last change the code for building facets was rewritten. And
for performance on the facets creation I pushed higher version
dependencies for MARC::File::XML and MARC::Record (we rely on
MARC::Field->as_string).

* Calls to MARC::Record->new_from_xml and MARC::Record->new_from_usmarc
are wrapped with eval for catching problems [2].

* As of bug 3087, UNIMARC uses the 'unimarc' record syntax. this case is
  correctly handled.
* As of bug 7818 misc/migration_tools/rebuild_zebra.pl behaves like:

- bib_index_mode (defaults to 'grs1' if not specified)
- auth_index_mode (defaults to 'dom')

here we do exactly the same.

To test:
 - prove t/db_dependent/Search.t should pass.
 - Searching should remain functional.
 - Indexing and searching for a big record should work (that's what the
   unit tests do).
 - Test an index scan search (on the staff interface):
    Search > More options > Check "Scan indexes".
 - Enable 'itemBarcodeFallbackSearch' and try to circulate any word, it
   shouldn't break.
 - Searching for a biblio in a new subscription shouldn't break.
 - Running bulkmarcimport.pl shouldn't break.
 - And so on... for the rest of the .pl files.

[1] http://search.cpan.org/~mirk/Net-Z3950-ZOOM/lib/ZOOM.pod#render()
[2] a record that cannot be parsed by MARC::Record is simply skipped (bug 10684)

Sponsored-by: Universidad Nacional de Cordoba
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-02-28 19:50:09 +00:00
Galen Charlton
5920ca6fa0 Bug 11389: reenable Pg as a DB scheme that Koha can connect to
This patch restores the ability to request a DBI database handle
or a DBIx::Class schema object connected to a PostgreSQL database.

To address the concerns raised in bug 7188, only "mysql" and "Pg"
are recognized as valid DB schemes.  If anything else is passed
to C4::Context::db_scheme2dbi or set as the db_scheme in the Koha
configuration file, the DBD driver to load is assumed to be "mysql".

Note that this patch drops any pretense of Oracle support.

To test:

[1] Apply patch, and verify that the database-dependent tests
    pass when run against a MySQL Koha database.
[2] To test against PostgreSQL, create a Pg database and
    edit koha-conf.xml to set db_scheme to Pg (and adjust
    the other DB connection parameters appropriately).  The
    following tests should pass, at minimum:

    t/Context.t
    t/db_dependent/Koha_Database.t

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as described, some additional notes:

- Installed Postgres following
  http://wiki.ubuntuusers.de/PostgreSQL
- Created a database user koha
- Created a database koha
- Changed the koha-conf.xml file
    <db_scheme>Pg</db_scheme>
    <database>koha</database>
    <hostname>localhost</hostname>
    <port>5432</port>
    <user>koha</user>
    <pass>xxxx</pass>
- Installed libdbd-pg-perl
- Ran the web installer until step 3 everything looked ok
  Step 3 complains:
    Password for user koha: psql: fe_sendauth: no password supplied
- Both t/Context.t and t/db_dependent/Koha_Database.t pass

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2014-01-13 20:56:14 +00:00
Galen Charlton
71b7d56298 Bug 10277: (follow-up) if no userenv is set, act like a superlibrarian
This patch fixes an error caught by t/db_dependent/Acquisition.t, and
adjusts C4::Context::IsSuperLibrarian() to return true if no
userenv is set. This is done on the basis that if no userenv is set,
calls to C4::Context routines are being made from a command-line script,
and if you have access to the command line of a running Koha instance,
you implicitly already have better than superlibrarian access.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-12-30 18:50:04 +00:00
Jonathan Druart
3e866d216d Bug 10277: Add UT for C4::Context::IsSuperLibrarian
Note that I modify the return value. Before this patch, it returned an
empty string or 1. Now it returns 0 or 1.

Test plan:
- same as the original patch
- verify that unit tests pass:
    prove t/Context.t

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script, including new tests.
Checked the code and tested superlibrarian behaviour in some places:

moremember.pl:
With IndyBranches only superlibrarian can delete borrowers from
other branches. Accessing the borrower with a direct link.
OK

C4/Members.pm
With IndyBranches only superlibrarian can search for borrowres
from other branches.
OK

tools/holidays.pl
With IndyBranches only superlibrarian can edit holidays for other
branches.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-12-30 15:47:46 +00:00
fb4025b67b Bug 10277 - Add C4::Context->IsSuperLibrarian()
The method of checking the logged in user for superlibrarian privileges
is obtuse ( $userenv && $userenv->{flags} % 2 != 1 ) to say the least.
The codebase is littered with these lines, with no explanation given. It
would be much better if we had one subroutine that returned a boolean
value to tell us if the logged in user is a superlibrarian or not.

Test Plan:
1) Apply this patch
2) Verify superlibrarian behavior remains unchanged

Signed-off-by: Joel Sasse <jsasse@plumcreeklibrary.net>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Comments on second patch.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-12-30 15:47:23 +00:00
Chris Cormack
398b9dfdbd Bug 8798: moving code to Koha::Database and adding tests
- Fixing a bug .. ping does not exist we need to use connected

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-14 21:08:02 +00:00
Elliott Davis
9f2f0dbc4a Bug 8798: (follow-up) remove tabs
Signed-off-by: wajasu <matted-34813@mypacks.net>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-14 21:08:02 +00:00
Elliott Davis
8a480ad75d Bug 8798: DBIx::Class base classes for all Koha tables
* Added base class files for all tables in koha using
DBIx::Class::Schema::Loader.
* Added a (very basic) test file for C4::Context
* Also added dependencies in required files.

To Test:

[1] Install patch
[2] Make sure you can still connect to Koha
[3] You may optionally run this test script:

        use Koha::Database;
        use Data::Dumper;
        my $db = Koha::Database->new();
        my $schema = $db->schema();
        print Dumper($schema->resultset("Borrower"));

    If you run this file you should get a DBIx dump of the borrowers table.

Signed-off-by: wajasu <matted-34813@mypacks.net>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-14 21:07:24 +00:00
Galen Charlton
b4c2ee3d9d Bug 10325: (follow-up) restore case-insensivity of syspref cache
Case-insensitivity of the system preference cache was introduced in
the patch for bug 6132.  This patch corrects some breakage that
occurred.  Longer-term, IMO a hard look needs to be taken at
using a case-insensitive collation for syspref codes, and coded
values in general.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-09-08 05:08:51 +00:00
47bd795b82 Bug 10325 - Allow system preferences to be overridable from koha-httpd.conf
For Koha installations with multiple OPAC URLs, It would be nice to be
able to override systeprefs from the http conf file. Case in point,
a library wants to have two separate opacs, one the is only viewable
from within the library that allows patrons to place holds, and a second
public one that does not. In this case, overriding the system preference
RequestOnOpac would accomplish this simply, and with no ill affects.

This feature would of course be should only be used to override
cosmetic effects on the system, and should not be used for system
preferences such as CircControl, but would be great for preferences
such as OpacStarRatings, opacuserjs, OpacHighlightedWords and many
others!

Test Plan:
1) Apply this patch
2) Disable the system pref OpacHighlightedWords
3) Do a seach in the OPAC, not the term is not highlighted
4) Edit your koha-http.conf file, add the line
     SetEnv OVERRIDE_SYSPREF_OpacHighlightedWords "1"
   to your koha-http.conf file's OPAC section.
   Also add the line
     SetEnv OVERRIDE_SYSPREF_NAMES "OpacHighlightedWords"
   to the Intranet section
5) Restart your web server, or just reload it's config
6) Do a seach, now your search term should be highlighted!
7) From the intranet preference editor, view the pref,
   You should see a warning the this preference has been overridden.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-09-08 02:09:53 +00:00
Galen Charlton
a7eb34f2c8 Revert "Bug 6554 - make Koha internally utf-8 clean"
This reverts commit d542740ab8.

Rolling back bug 6554 work until we have more comprehensive tests.

Conflicts:

	opac/opac-search.pl
2013-04-29 15:12:32 -07:00
Galen Charlton
4ba31d2e28 Revert "Bug 6554 Fix error caused by modifying $_ in a map"
This reverts commit e98fe7e322.

Rolling back bug 6554 work until we have more comprehensive tests.
2013-04-29 15:10:45 -07:00
Colin Campbell
e98fe7e322 Bug 6554 Fix error caused by modifying $_ in a map
Error 'Modification of a read-only value attempted' triggered
on login because of manipulation of $_ in the map

Moved the mod to a loop as recommended in the doc for map

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-04-18 09:26:22 -04:00
Dobrica Pavlinusic
d542740ab8 Bug 6554 - make Koha internally utf-8 clean
In current implementation (mostly commented out in this patch)
uses heuristic to guess which strings need decoding from utf-8
to binary representation and doesn't support utf-8 characters
in templates and has problems with utf-8 data from database.

With this changes, Koha perl code always uses utf-8 encoding
correctly. All incomming data from database is allready
correctly marked as utf-8, and decoding of utf8 is required
only from Zebra and XSLT transfers which don't set utf-8 flag
correctly.

For output, standard perl :encoding(utf8) handler is used
so it also removes various "wide character" warnings as side-effect.

Test scenario:
1. make sure that you have utf-8 characters in your biblio
   records, patrons, categories etc.
2. try to search records on intranet and opac which contain
   utf-8 characters
3. install language which has utf-8 characters, e.g. uk-UA
   dpavlin@koha-dev:/srv/koha/misc/translator(bug_6554) $
   PERL5LIB=/srv/koha/ perl translate install uk-UA
4. switch language to uk-UA and verify that templates
   display correctly
5. test search and Z39.50 search and verify that caracters
   are correct

Signed-off-by: Owen Leonard <oleonard@myacpl.org>

I followed the test plan, adding utf-8 characters to library names,
patron categories, titles, and authorized values. I tried the uk-UA
translation and everything looked good.

When performing Z39.50 searches for titles containing utf-8 characters I
got results which were still occasionally contaminated with dummy
characters [?] but I assume this is Z39.50's fault not the patch's.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Already signed, add mine.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-04-01 18:51:49 -04:00
Jared Camins-Esakov
b6b5f2b59a Revert "Merge branch 'bug_8089' into 3.12-master"
This reverts commit af89f98a7a, reversing
changes made to cc49dc70fb.

The changes made by bug 8089 caused problems with the
t/db_dependent/Context.t unit test. Given the proximity of feature
freeze and release I am exercising my RM privilege and reverting rather
than seeking to fix the problems right this moment.
2013-03-21 21:06:30 -04:00
Jared Camins-Esakov
f2c8262fae Bug 8089: Cache sysprefs with Koha::Cache::Object
This patch replaces the %sysprefs hash variable with a
Koha::Cache::Object-tied hash that allows the user to cache sysprefs
in memcached, fastmmap, or memory.

To test:
1) Enable the cache as described on bug 9434.
2) Try viewing some pages in the OPAC and staff client. If the syspref
   caching isn't working, you'll be redirected to the web installer
   (in the staff client) or be shown the maintenance page (in the OPAC).
3) Disable the cache by unsetting the CACHING_SYSTEM environment
   variable.
4) Try viewing some pages in the OPAC and staff client. If the syspref
   caching isn't working, you'll be redirected to the web installer
   (in the staff client) or be shown the maintenance page (in the OPAC).

Note that this will not improve performance. It does, however, eliminate
the problem of sysprefs not getting updated in all threads in
multi-threaded environments.

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>

Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-03-21 19:15:31 -04:00
Bernardo Gonzalez Kriegel
9205e732fb Bug 9827: remove 'insecure' system preference
This patch removes 'insecure' system preference.

Also removes remaining code that make use of
the preference. It's broken anyway.

Only remains a reference in POD of C4/Boolean.pm

To test:
1) If you like, enable 'insecure' syspref. Broken system.
WARN: be prepared to revert value in database.

2) Apply the patch

3) Run updatedatabase.pl

4) Check that Staff login proceeds as usual.

5) Check that 'insecure' syspref is no more.

Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Amended patch: Remove 2 occurrences of insecure (in comment only)
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-03-19 21:46:34 -04:00
Jared Camins-Esakov
397d0f15b7 Bug 9832: avert software error with QP enabled and misconfigured
QueryParser requires a configuration file. If koha-conf.xml does not
point to that configuration file, you will get a software error when
you try to search with QueryParser enabled. Until this patch.

To test:
1) Comment out the <queryparser_config> line in your koha-conf.xml
2) Turn on UseQueryParser
3) Do a search, watch it explode
4) Apply patch
5) Do search again, watch it work
6) Sign off on patch

Signed-off-by: Mirko Tietgen <mirko@abunchofthings.net>
Works as expected, nothing explodes anymore

Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-03-19 08:19:30 -04:00
Jared Camins-Esakov
b086fcab9a Bug 9239: Add syspref to enable QueryParser
In light of the experimental nature of the QueryParser code at this
juncture, it behooves Koha to have the option of disabling QP, at least
for the purpose of comparing results between QP-enabled and QP-disabled
searches. This patch adds a UseQueryParser syspref to do just that. In
cases where the user enables QP without having QP installed, however,
there will be no dramatic failure, and Koha will simply fall back to
using the non-QP search code.

In order to reduce the overhead added by the introduction of QueryParser,
this patch adds a C4::Context->queryparser accessor to a static
QueryParser object which is initialized the first time it is required
and then reset thereafter.

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Elliott Davis <elliott@bywatersolions.com>
Fixed merge conflict in sysprefs.sql and updatedatabase.sql

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Fixed merge conflict in sysprefs.sql und updatedatabase.pl
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-03-16 21:32:32 -04:00
Katrin Fischer
d8d2572257 Bug 9587 : QA follow up - fix one tabulation character
Makes QA script happy.

Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-02-24 10:15:56 -05:00
d03c8443a4 Bug 9587 Follow up to fix the problem Katrin identified
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>

Comment: Solve that problem, but now koha-qa complains about tabs
in C4/Context.pm.

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Tests done:
1) System preference 'Persona' added correctly.
2) Persona off, normal login still possible
3) Persona on, Persona login works
4) Persona logout works
5) normal login still possible
6) normal logout still possible

Persona is off by default and uses the primary email address
from the patron account.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-02-24 10:15:55 -05:00
Jared Camins-Esakov
38f8bf4776 Revert "Bug 7167: Set final updatedatabase.pl version"
This reverts commit c34017336c.
2012-12-27 14:02:50 -05:00
Jared Camins-Esakov
c34017336c Bug 7167: Set final updatedatabase.pl version 2012-12-27 11:55:22 -05:00
Jared Camins-Esakov
a6c7ba8519 Bug 9191: updatedatabase.pl should only run updates up to the current version
Check whether a given update should be run when passed the proposed
version number. The update will always be run if the proposed version
is greater than the current database version and less than or equal to
the version returned by C4::Context->final_linear_version (initially set
to be equal to the version in kohaversion.pl). The update is also run if
the version contains XXX, though this behavior will be changed following
the adoption of non-linear updates as implemented in bug 7167.

To test:
1) Make sure that the first example database update added by this patch
   in installer/data/mysql/updatedatabase.pl has a version number one
   greater than the version of Koha you have installed.
2) Set the Version syspref back one version behind your current version.
3) Navigate to the main page of the staff client, and log in to the
   installer.
4) Confirm that the update page claims to have rerun the previous update
   and has displayed the log message:
   "Upgrade to 3.11.00.XXX done (Bug 9191: You should see this)"
   but not the log message:
   "Upgrade to [version number] done (Bug 9191: You shouldn't see this)"

Note: the sample database revisions will be removed by the RM before
this patch is pushed to master.

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
works great

Signed-off-by: Elliott Davis <elliott@test.bywatersolutions.com>
Works as expected.  Test plan is great. Code looks to be adherent to standards.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2012-12-11 08:19:00 -05:00
Chris Nighswonger
cb6870d8d1 Bug 9085 - Installer will not run with DEBUG set
This is due to raising all db errors combined with the fact that attempts
to make selections from a non-existent systempreferences table raises db errors.

The installer should run even with DEBUG set, otherwise dropping and recreating
a development database becomes somewhat of a chore. This patch adds code to
check for the existence of the systempreferece table. If the table exists, then
RaiseError is enabled on the database handle. Otherwise it is disabled to allow
the installer to run without throwing an error.

Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
The installer can now be run both with and without DEBUG set.

Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2012-11-20 06:50:08 -05:00
Jared Camins-Esakov
4b65389d5e Bug 9005: Allow user to disable syspref cache
Because C4::Context uses an in-memory hash for caching sysprefs,
changing a syspref under a multi-threaded persistent environment
requires a server restart. This patch makes it possible disable
the syspref cache.

To test:
1) If you are using a multi-threaded persistent server (Starman, etc.),
   change a syspref and note that the effects of the syspref change may
   or may not be visible on any given request before applying this patch.
   You will need to choose a syspref with obvious effects that can be
   seen by simply refreshing the page. I recommend enabling or
   disabling additional languages in the OPAC, since you can refresh the
   page a dozen times and reasonably expect to see the new behavior you
   set only 1/n of the time.
2) Apply patch.
3) Add "C4::Context->disable_syspref_cache();" to your koha.psgi file
4) Repeat step 1, noting that you never see the stale behavior.
5) Run test at t/db_dependent/Context.t.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
All tests passed.

Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2012-11-08 12:41:10 -05:00
084367f6cb Bug 3087 Fix Z39.50 server to return the correct record syntax
Modify Makefile.PL and Zebra configuration files in order to parametrized
biblio record type returned by Zebra Z39.50 server.

How to test:

- Test with a MARC21 and a UNIMARC DB
- Do a new installation
- Search from OPAC
- Search from a Z39.50 client like yaz-client: syntax = MARC21/UNIMARC must be
  choosed
- It was working for MARC21: it continues to work
- It wasn't working for UNIMARC: it works now, both in OPAC and from a Z39.50
  client

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Works fine for MARC21. Frederic looked at UNIMARC. Magnus looked at NORMARC.
GRS1 works okay for me. I still have issues with DOM, but they are not directly related to changes in this patch.
A followup is still needed for packaging (debian/templates).

Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-10-22 14:12:22 +02:00
809b543d27 Bug 6679 -[SIGNED-OFF] fix 12 perlcritic violations in C4/Context.pm
"return" statement with explicit "undef" at line 486, column 5.  See page 199 of PBP.  (Severity: 5)

Subroutine prototypes used at line 548, column 1.  See page 194 of PBP.  (Severity: 5)

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-09-20 11:39:37 +02:00
Jared Camins-Esakov
7ad5e203da Bug 2060: Update command line MARC import scripts
Expose authority import functionality to the command line import
scripts, and rename them from commit_biblios_file.pl and
stage_biblios_file.pl to commit_file.pl and stage_file.pl.

To test (note that these instructions assume you have a MARC21
installation and are using the provided sample file):
1. Find a file of authorities (a sample file with MARC21 authorities
   is attached to bug 7475) and download it to your server
2. Stage the file using the following command (replace <filename> with
   the name of the file you saved in step 1):
   > misc/stage_file.pl --file <filename> --authorities
3. Note the batch number the script assigns to your batch
4. Commit the records using the following command (replace <batchnumber>
   with the batch number you made note of in step 3):
   > misc/commit_file.pl --batch-number <batchnumber>
5. Index the authorities Zebraqueue (or wait)
6. Confirm that the new authorities appear.
7. Create a matching rule with the following settings:
   Code: AUTHTEST
   Description: Personal name main entry
   Match threshold: 999
   Record type: Authority record
   Search index: Heading-main
   Score: 1000
   Tag: 100
   Subfields: a
   Offset: 0
   Length: 0
   (note the ID of this matching rule)
8. Stage the authority file again, this time using the following
   command:
   > misc/stage_file.pl --file <filename> --authorities \
     --match <matchingrule>
7. Revert the import with the following command:
   > misc/commit_file.pl --batch-number <batchnumber> --revert
8. Index the authorities Zebraqueue (or wait)
9. Confirm that the records have been removed
10. Import an authority record with the Stage MARC/Manage staged MARC
    tools in exactly the way you would for a bibliographic record,
    but choose "Authority" instead of "Bibliographic" for the record
    type.

Signed-off-by: Elliott Davis <elliott@bywatersolutions.com>

Testing plan delivers as it should.

Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on latest master 11 September 2012
2012-09-19 17:16:18 +02:00
Paul Poulain
8449b14dcb Revert "Bug 8089: Use Koha::Cache for all caching"
This reverts commit 215abc8024.

The 3 patches for bug 8089 have been reverted, because they break
jenkins & Koha.
A follow-up has been provided, but it does not solve the problem on my
test server, it just changes the error message.

After a discussion with jared, Dobrica should work on another patch, so
the best option is to revert.
2012-09-12 14:12:41 +02:00
Paul Poulain
04881d9265 Revert "Bug 8089: Cache sysprefs using Koha::Cache"
This reverts commit 499f44fa1d.
2012-09-12 13:45:51 +02:00
Jared Camins-Esakov
499f44fa1d Bug 8089: Cache sysprefs using Koha::Cache
Cache sysprefs using Koha::Cache in a way that is safe even for
caching-related sysprefs. This lays the groundwork for removing
caching configuration from the httpd.conf and configuring it
using sysprefs.

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
2012-09-07 16:28:31 +02:00
Jared Camins-Esakov
215abc8024 Bug 8089: Use Koha::Cache for all caching
1. Replace all instances of memoize_memcached with appropriate calls
into Koha::Cache:
* reports/guided_reports.pl
* C4::Biblio::GetMarcStructure
* C4::Languages::getFrameworkLanguages
* C4::Languages::getAllLanguages
* C4::SQLHelper::GetPrimaryKeys
* C4::SQLHelper::_get_columns

2. Replace all references to memcached with the appropriate calls into
Koha::Cache in C4::Context.

Test plan :
* have DEBUG env set to 1
* reach addbiblio page to test the patch in Biblio.pm, or setup more than 1
  language
* you should see in the logs that you're reading and writing from cache
* run the test suite twice both with and without the following environment
  variables set:
export MEMCACHED_SERVERS=127.0.0.1:11211
export MEMCACHED_NAMESPACE=KOHA
export CACHING_SYSTEM=memcached

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>

I'm unsure about some of the caching times 10000 is a long long time,
but other than that, works fine.
2012-09-07 16:28:29 +02:00
Chris Cormack
509d673f10 Bug 7941 : Fix version numbers in modules
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-06-11 17:29:38 +02:00
Colin Campbell
7716fce253 Bug 5549 : Add a timezone object to C4::Context
Will be required by rolling loans to get correct times
NB the setting of the TZ in mysql has not been changed as
this appears to rely on $ENV{TZ} being undefined
2012-03-20 12:08:20 +13:00
Paul Poulain
6a520f7456 Bug 6193 follow-up of the follow-up = reindentation only 2012-02-21 13:16:11 +01:00
52afe06ddd Bug 6193 - Follow up: use SetEnv and remove memcached from koha-conf.xml
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Confirmed that memcached is still being used after the memcached configuration
in koha-conf.xml was removed, and the following two lines were added to
both virtual hosts in koha-httpd.conf:
SetEnv MEMCACHED_SERVERS "127.0.0.1:11211"
SetEnv MEMCACHED_NAMESPACE "KOHA"
2012-02-20 23:24:02 +01:00
Paul Poulain
d7a51d83c1 Merge remote-tracking branch 'origin/new/bug_6132' 2012-01-16 14:53:08 +01:00