Commit graph

424 commits

Author SHA1 Message Date
Olli-Antti Kivilahti
bf06e42414
Bug 13937: Fix the daemon log path.
directory _LOG_DIR_/logs does't exists.

All other log4perl -logs are out to _LOG_DIR_

Thank you for the good work Ere!

Signed-off-by: Stefan Berndtsson <stefan.berndtsson@ub.gu.se>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-10-08 14:40:12 +01:00
Ere Maijala
5af4cc2e65
Bug 13937: Add support for Z39.50 responder to the Debian scripts
Signed-off-by: Stefan Berndtsson <stefan.berndtsson@ub.gu.se>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-10-08 14:40:07 +01:00
Ere Maijala
ac28d8043d
Bug 13937: Add support for SRU and Elasticsearch
Splits Session to GenericSession and ZebraSession where Generic supports any
search backend via the SearchEngine classes and Zebra maintains the direct
channel to the Zebra server.

Adds config files required for mapping BIB-1 attributes to Koha search fields
and SRU indexes to BIB-1 attributes.

Adds PODs.

Sponsored-by: National Library of Finland
Signed-off-by: Stefan Berndtsson <stefan.berndtsson@ub.gu.se>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-10-08 14:39:53 +01:00
Jesse Weaver
f089d393dc
Bug 13937: Add a Z39.50 daemon that can inject item status MARC subfields
This creates a new daemon, misc/z3950_responder.pl, which can respond to
Z39.50 requests. By default, it just proxies searches to Zebra.

If desired, however, it can also add a subfield to the item tags on
outgoing records with a textual description of the item's status
(checked out, lost, etc.). This is useful for certain ILL systems. These
strings can be translated using the 'Z3950_STATUS' authorized value.

Test plan:
  1) Start the Z39.50 server using `perl misc/z3950_responder.pl`.
  2) Connect to the server using `yaz-client 127.0.0.1:9999/biblios`.
  3) Run a search, such as `find @attr 1=1016 book`.
  4) Fetch the results both one at a time with `show 1` and in a batch
     using `show 1+5`.
  5) Turn on MARCXML using `format xml` and `elements marcxml`, and
     verify that the records are still correctly fetched.
  6) Enable the item status subfield by restarting the server with the
     option `--add-item-status=k`.
  7) Search for and fetch records, and verify that a $k subfield is
     added to the item tags as appropriate. It should show some
     combination of "Checked Out", "Lost", "Not For Loan", "Damaged",
     "Withdrawn", "In Transit", or "On Hold" as appropriate, or
     "Available".
  8) Add an authorized value named "Z3950_STATUS" with any of the keys
     "AVAILABLE", "CHECKED_OUT", "LOST", "NOT_FOR_LOAN", "DAMAGED",
     "WITHDRAWN", "IN_TRANSIT" or "ON_HOLD", and verify that their
     descriptions are used instead of the default values above.

Signed-off-by: George Williams <george@nekls.org>
Signed-off-by: Stefan Berndtsson <stefan.berndtsson@ub.gu.se>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-10-08 14:39:30 +01:00
Olli-Antti Kivilahti
26c0b23764
Bug 17851: Add elasticsearch config to koha-conf.xml
Missing installer for debian packages.

Sponsored-by: Koha-Suomi Oy
Signed-off-by: Johanna Raisa <johanna.raisa@gmail.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-10-07 14:09:10 +01:00
491a2e858f
Bug 23722: Document allow_empty_passwords in the example SIP config file
The example SIP config file in Koha is kind of a kitchen sink document where
all available SIP2 options should be demonstrated.
The allow_empty_passwords flag should be included in it.

Signed-off-by: Liz Rea <wizzyrea@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-10-03 10:28:24 +01:00
023cfa76ba
Bug 20954: (QA follow-up) Rename syspref-overrides to syspref_overrides to maintain existing convention
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-08-13 11:48:12 +01:00
9cbaec2a94
Bug 20954: Add ability to set syspref overrides in SIP as we can in apache
We should be able to set system preference overrides for SIP in a
similar manner that we do in Apache. It would be great if we could
specify those overrides on both a config level, and login level basis.

Test Plan:
 1) Apply this patch
 2) Start your SIP server
 3) Enable the syspref AllFinesNeedOverride
 4) Find or create a patron with a small fine ( less than noissuescharge )
 5) Attempt to check out an item to the patron, it should fail
 6) Add the global syspref override from the bottom of the example SIP config file
 7) Restart your SIP server
 8) Attempt to check out an item to the patron again, this time it should work
 9) Now, add the login level syspref override section as it appears in
    the eaxmple SIP config file. Make sure to add it to the login you are using
10) Attempt to check out another item to the patron, this time is should
    again fail

Signed-off-by: Liz Rea <wizzyrea@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-08-13 11:47:36 +01:00
14af35852e
Bug 20292: Filter/censor info sent via SIP
Some libraries would like to limit the amount of personal information a SIP server sends
to arbitrary parties on a per-login basis.

Test Plan:
1) Add a new key/value pair to one of your existing login stanzas in your SIP config file
   For example: hide_fields="BD,BE,BF,PB"
2) Restart SIP
3) Send a SIP message that would normally return those fields ( in this example, a Patron Information Request )
4) Note the response has had those fields removed

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Christopher Davis <tubaclarinet@protonmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-06-24 15:33:48 +01:00
3599e57bf6
Bug 15704: Split up Zebra indexing of RDA 264 information
To test:
1 - Add a record with a unique publisher "Supercalifragilistic" in the
264 b field
2 - Search for the value
3 - Record not found
4 - Apply patch (may need ot copy the .xml file into koha install)
5 - Reindex all the things
6 - Search for the value
7 - Success!

Signed-off-by: Felicia Martin <felicia.martin@dncr.nh.gov>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-06-24 14:53:18 +01:00
David Nind
054094835d
Bug 23130: Fix alternative mana server URL in etc/koha-conf.xml
If the alternative mana KB server URL in etc/koha-conf.xml is used it
causes an error message like 'malformed JSON string, neither tag,
array, object, number, string or atom, at character offset 0 (before
"<html>\r\n<head><tit...") at /usr/share/perl5/JSON.pm line 171.' when
submitting an account creation request, and the account creation
request fails.

This patch updates the alternative mana KB server URL in
etc/koha-conf.xml from http://mana-test.koha-community.org to
https://mana-test.koha-community.org (the URL should start with https
instead of http). If the updated URL is used the account creation
request succeeds without causing any error messages.

To test:
1) Add <mana_config>http://mana-test.koha-community.org</mana_config> to
   /etc/koha/sites/<instancename>/koha-conf.xml (see etc/koha-conf.xml
   in the Koha code repository for an example of where to add this;
   <instancename> if using koha-testing-docker is kohadev).
2) Clear memcached and restart services so that the changes to your Koha
   instance configuration are recognised (if using koha-testing-docker
   run flush_memcached and then restart_all).
3) From the staff client home page go to Koha administration >
   Additional parameters > Share content with Mana KB.
4) Enable content sharing: change 'Use Mana KB for sharing content' to
   Yes and press Save.
5) Enter your first name, last name and email address in the Configure
   Mana KB section of the page and then click on 'Send to Mana KB'.
6) An error message is displayed, this may be something like:
   'malformed JSON string, neither tag, array, object, number, string
   or atom, at character offset 0 (before "<html>\r\n<head><tit...") at
   /usr/share/perl5/JSON.pm line 171.'
7) Change the mana server URL to https://mana-test.koha-community.org
   (see step 1), and repeat steps 2, 3 and 5.
8) Instead of an error message you should get 'You successfully created
   your Mana KB account. Check your mailbox and follow instructions.'

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-06-20 09:33:11 +01:00
ef0c31f735
Bug 23053: [follow-up] Same changes for UNIMARC
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-06-13 14:38:04 +01:00
2e8a6a4b09
Bug 23053: Add phrase index to authority Local-Number
To test:
 1 - Define a matching rule for authorities on field 001 index Local-Number
 2 - In koha-conf.xml raise the zebra_loglevels
     <zebra_loglevels>none,fatal,warn,request,info</zebra_loglevels>
 3 - Export some authorities using the tools->export data
 4 - Import those authorities
 5 - Note no matches found
 6 - View the zebra output log, you should see lots of error 114
 7 - Apply patch
 8 - Copy the indexdefs files to the installed versions
 9 - Reapply matchign rules to staged files
10 - Matches should now be found
11 - Logs should not have errors

Signed-off-by: Claire Gravely <claire.gravely@bsz-bw.de>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-06-11 08:37:40 +01:00
Alex Arnaud
8427b68414
Bug 22915: fix Mana URL in etc/koha-conf.xml
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-06-07 09:28:21 +01:00
Julian Maurice
9532a3edea Bug 22705: Change default value of cxn_pool to 'Static'
'Static' is the default value of Search::Elasticsearch and for a good
reason : it works in most cases, unlike the 'Sniff' option

To test:
1 - Apply patch
2 - Edit koha-conf.xml
3 - Add '<cxn_pool>Static</cxn_pool>' to the elasticsearch stanza
4 - Restart all the things!
5 - Reindex ES, it works
6 - Set SearchEngine to ES, try searching
7 - It works!

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2019-05-23 14:41:57 +00:00
d57239ca78 Bug 22787: Add mapping of letters "ů" and "Ů" to zebra definitions
Test plan:
0) Do not apply the patch
1) Add some word with character "ů" into metadata, for example author "Martinů, Bohuslav"
2) Try to search it with "Martinu" and you'll see you can't find it
3) Apply the patch
4) Copy file etc/zebradb/etc/word-phrase-utf.chr to your /etc/koha directory
    sudo cp etc/zebradb/etc/word-phrase-utf.chr /etc/koha/zebradb/etc/
5) koha-zebra --restart kohadev
6) koha-rebuild-zebra -f kohadev
6) try to search "Martinu" again - you should be able to find your record

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2019-05-07 11:00:13 +00:00
296e0ce224 Revert "Bug 15253: Add Koha::Logger based logging for SIP2"
This reverts commit 03c03ed8cd.

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2019-04-18 17:57:51 +00:00
03c03ed8cd Bug 15253: Add Koha::Logger based logging for SIP2
Now that we have Koha::Logger, we should use it in our SIP server. This
has the potential to make debugging SIP issue much easier. We should add
the userid for the sipuser to the namespace so we can allow for separate
files per sip user if wanted.

Also modifies the log4perl.conf to lazy-open filehandles to log files,
so the same config can be used with log-files needing different
permissions.

Test Plan:
1) Apply this patch set
2) Update the modififed log4perl.conf to your system
3) Restart your sip server
4) Tail your sip2.log, run some queries
5) Note you still get the same output messages as before, with the
   addition of the ip address and username ( if available )
   prefixing the message.

Based on original patches by Kyle Hall and additions by Olli-Antti
Kivilahti.

Signed-off-by: Liz Rea <wizzyrea@gmail.com>
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2019-04-18 11:37:27 +00:00
9f5a1bc7eb Bug 22043: (QA follow-up) Add parameter to control behavior
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2019-03-15 12:10:46 +00:00
3e63209fa8 Bug 22016: Always send CT field for SIP checkin, even if empty
Some SIP devices ( in this particular case, bin sorting machines from RFID Library Solutions ) require a CT field to be sent, even if that field is empty. Koha should be able to support this behavior.

Test Plan:
1) Apply this patch
2) Enable the new option ct_always_send for a SIP2 account
3) Restart SIP
4) Check in an item successfully via SIP, which will not be transferred
5) Note the response contains a CT field with no value

Sponsored-by: Pueblo City-County Library District

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jill Kleven <jill.kleven@pueblolibrary.org>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2019-02-27 09:14:21 -05:00
9cdac855c9 Bug 22014: Add ability to send "00" in SIP CV field on checkin success
Some SIP devices ( in this particular case, bin sorting machines from RFID Library Solutions ) require a checkin success to return a CV field of the value "00" rather than no CV field at all. Koha should be able to support this behavior.

Test Plan:
1) Apply this patch
2) Enable the new option cv_send_00_on_success for a SIP2 account
3) Restart SIP
4) Check in an item successfully via SIP
5) Note the response contains a CV field with the value '00'

Sponsored-by: Pueblo City-County Library District

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jill Kleven <jill.kleven@pueblolibrary.org>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2019-02-27 09:14:21 -05:00
Baptiste Wojtkowski
cb01b25330 Bug 17047: SQL reports management with Mana-KB
Includes:

* code factorization
Some code from subscription & Mana-KB has been factorized in order to speed-up next developments

* SytemPreferences:
Mana Activation:
    - add a value "no, let me think about it", that is the default value.
    - as long as this value is selected, messages ask if user want to activate it ( in Administration and Add-subscription(page 2) )
AutoShareWithMana
    - Add the syspref AutoShareWithMana: user can automatically share infos with Mana-KB (not set by default)

* Interface :
    - On mana-search, rows are now sorted by date of last import, then by number of users
    - Windows redesigned to improve the user experience

* New Feature : report a mistake.
    - people can now report an invalid data (wrong, obsolete,...)
    - if a data is reported as invalid many time, it will appear differently
    - Added few tooltip (to explain the fields last import, nb of users, to explain the new feature)
    - When reporting a data as invalid, a comment can also be added. Koha will then display comments related to data in result lists

* API (svc/mana)
    - add svc/mana/addvaluetofield: allows to ask mana incrementing a field of a resource
    - no hardcoding for resources in the code of api (api needs to be called with a ressourcename)

* New feature : SQL report sharing
    - Create Koha::Report.pm and Koha::Reports.pm, objects class for Reports
    - New feature: share reports with Mana-KB
    - New feature: search report in Mana-KB with keywords
    - New feature: load reports from Mana-KB

Test plan:
    1 - Apply Patch + update database
    2 - Copy the three lines about mana config in etc/koha-conf.xml in ../etc/koha-conf.xml (after <backupdir> for example)
        <!-- URL of the mana KB server -->
        <!-- alternative value http://mana-test.koha-community.org to query the test server -->
        <mana_config>https://mana-kb.koha-community.org</mana_config>
    3 - Check Mana syspref and AutoShareWithMana syspref are not activated
    4 - Search the syspref ManaToken and follow the instructions
    5 - subscriptions
      - Try create a new subscription for a first serial  => Mana-KB shouldn't show you anything (except if the base hase been filled)
      - Share this serial with Mana-KB (on the serial individual's page there must be a Share button)
      - Try to create a new subscription for serial nr1 => a message should appear when you click on "next", click on "use", the fields should automaticaly appear
      - Activate AutoShareWithMana => Subscriptions
      - Create a new subscription for a second serial
      - There shouldn't be any Share button
      - Create a second subscription => the message should appear, click again on use

    6 - SQL Report
      - Create a new SQL report, without notes.
      - On the table with all report (reports > use saved), there should be the action "Share"
      - If you click on share, you have an error message
      - Create a new report, with a title and notes longer than 20 characters
      - You  can share it with mana => you will have a success message
      - On (report > use saved), there must be a message inviting you to search on Mana-KB for more results, enter a few word from title, notes, type of  the report you shared, it should appear. You can use it, it will load it into your report list.

    7 - Report mistakes.
      - On any table containing Mana-KB search results, you can report a mistake and add a comment.

    8 - For each previous test, try to send wrong data, to delete the security token, to send nothing: it should show a correct warning message.

Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Rebased-by: Alex Arnaud <alex.arnaud@biblibre.com> (2018-07-04)
Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2019-01-23 14:39:26 +00:00
5c313bc554 Bug 22073: Add to diacritics mappings for Ž and ž
To test:
1 - Find or add a record by author Slavoj Žižek
2 - Search for 'Zizek'
3 - No results
4 - Apply patch
5 - Reindex
6 - Search again
7 - Success!

https://bugs.koha-community.org/show_bug.cgi?id=22073

Signed-off-by: Eric Phetteplace <ephetteplace@cca.edu>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2019-01-04 03:13:38 +00:00
cfc6c112d3 Bug 20078: (follow-up) Prevent conflict from attribute duplication
To test:
1 - Apply first patch
2 - Attempt searching by arp, no results
3 - Apply this patch
4 - Copy bib1.att and ccl.properties to the correct locations
5 - Restart zebra
6 - Rebuild indexes
7 - Search agian, success!

Signed-off-by: Margie Sheppard - Central Kansas Library System CKLS <msheppard@ckls.org>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2018-11-06 12:22:53 +00:00
Andrew Isherwood
8e8f26f2cb Bug 20996: (follow-up) Add <branch> to conf tpl
Since we now require the <branch> block, we should add it to the config
templates

Signed-off-by: Magnus Enger <magnus@libriotech.no>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2018-11-02 10:33:04 +00:00
02222d2e9b Bug 14302: (QA follow-up) Remove remaining file
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2018-08-31 11:24:21 +00:00
bee097b39b Bug 14302: Remove GRS1 specific code
Remove:
- BIB_INDEX_MODE and AUTH_INDEX_MODE env var
- bib_index_mode and auth_index_mode options from scripts
- Warnings from about page, just kept one if zebra_bib_index_mode or
zebra_auth_index_mode still exist in config and are set to grs1

Test plan:
- Install Koha from src
- Install Koha from pkg
- Read the code, carefully!

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Rebased

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2018-08-31 11:24:20 +00:00
cc87e0a458 Bug 14302: Remove GRS1 related files
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2018-08-31 11:24:19 +00:00
c6b50f0bac Bug 21031: Apache Rewrite rules don't work for API when using anything but Debian package Plack configuration
The Rewrite rules for Apache don't work unless you're using
debian/templates/apache-shared-opac-plack.conf or
debian/templates/apache-shared-intranet-plack.conf.

This patch fixes the Rewrite rules for the non-Plack Debian
Apache configuration templates as well as the standard
Apache configuration file that comes with Koha.

__BEFORE APPLYING__
1. Visit /api/v1/app.pl/api/v1/spec on your git dev install
2. This should display a large page of JSON
3. Visit /api/v1/spec on your git dev install
4. This should generate a 404 error

__APPLY PATCH__

__AFTER APPLYING__
5. Visit /api/v1/app.pl/api/v1/spec on your git dev install
6. This should display a large page of JSON
7. Visit /api/v1/spec on your git dev install
8. This should display a large page of JSON (identical to
the one from earlier steps)

Signed-off-by: Ere Maijala <ere.maijala@helsinki.fi>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Passed QA with few notes posted separately to Bugzilla.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2018-08-02 10:23:08 -03:00
14166c4a8c Bug 18322: Update xslt for NORMARC and UNIMARC
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2018-07-18 17:42:49 +00:00
f37cac60e0 Bug 18322: (follow-up) Add generated xsl
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2018-07-18 17:42:49 +00:00
17a354f5e8 Bug 18322: Add a facet for ccode fields to Zebra
This patch adds the index definitions for zebra faceting of ccode in
koha for marc21, normarc and unimarc.

We also add lines to the templates to expose the new facet and enable
non-zebra faceting for ccode too.

Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2018-07-18 17:42:48 +00:00
Ere Maijala
133df5e912 Bug 20073: Move Elasticsearch configs to yaml files and improve the default settings.
Improvements:
1) Index settings moved from code to etc/searchengine/elasticsearch/index_config.yaml. An alternative can be specified in koha-conf.xml.
2) Field settings moved from code to etc/searchengine/elasticsearch/field_config.yaml. An alternative can be specified in koha-conf.xml.
3) mappings.yaml has been moved from admin/searchengine/elasticsearch to etc/searchengine/elasticsearch. An alternative can be specified in koha-conf.xml.
4) Default settings have been improved to remove punctuation from phrases used for sorting etc.
5) State variables are used for storing configuration to avoid parsing it multiple times.
6) A possibility to reset the fields too has been added to the reset operation of mappings administration.
7) mappings.yaml has been moved from admin/searchengine/elasticsearch to etc/searchengine/elasticsearch.
8) An stdno field type has been added for standard identifiers.

To test:
1) Run tests in t/Koha/SearchEngine/Elasticsearch.t
2) Clear tables search_fields and search_marc_map
3) Go to admin/searchengine/elasticsearch/mappings.pl?op=reset&i_know_what_i_am_doing=1
4) Verify that admin/searchengine/elasticsearch/mappings.pl displays the mappings properly, including ISBN and other standard number fields.
5) Index some records using the -d parameter with misc/search_tools/rebuild_elastic_search.pl to recreate the index
6) Verify that you can find the records
7) Put <elasticsearch_index_mappings>non_existent</elasticsearch_index_mappings> to koha-conf.xml
8) Verify that admin/searchengine/elasticsearch/mappings.pl?op=reset&i_know_what_i_am_doing=1 fails because it can't find non_existent.
9) Copy etc/searchengine/elasticsearch/mappings.yaml to a new location and make elasticsearch_index_mappings setting in koha-conf.xml point to it.
10) Make a change in the new mappings.yaml.
11) Clear table search_fields (mappings reset doesn't do it yet, see bug 20248)
12) Go to admin/searchengine/elasticsearch/mappings.pl?op=reset&i_know_what_i_am_doing=1
13) Verify that the changes you made are now visible in the mappings UI

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Bug 20073: Move Elasticsearch yaml files back to admin directory

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2018-06-22 14:37:56 +00:00
9b39b9d20b Bug 20259: Shorter JS and CSS rewrite rule
Bug 20187 has changed the JS and CSS rewrite rules to :
RewriteRule ^(.*)_[0-9][0-9]\.[0-9][0-9][0-9][0-9][0-9][0-9][0-9].js$ $1.js [L]
RewriteRule ^(.*)_[0-9][0-9]\.[0-9][0-9][0-9][0-9][0-9][0-9][0-9].css$ $1.css [L]

This patch changes this rules using [0-9]{N} and fusion in one rule.
And espaces the dot in extension js and css.

Test plan :
1) Go to intranet and opac
2) Check CSS and JS are doing well
3) Apply patch changes on our Apache configuration
4) Reload intranet and opac pages (Ctrl + F5)
5) Check CSS and JS are doing well

Signed-off-by: Charles Farmer <charles.farmer@inLibro.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2018-06-22 13:30:46 +00:00
cff62796ea Revert "Bug 20073: Move Elasticsearch configs to yaml files and improve the default settings."
This reverts commit f489d2034b.

This commit breaks the install process when using debian packages.
Reverting as we are very close to the 18.05.00 release

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2018-05-21 11:02:16 -03:00
fdb5d0ca1a Bug 20428: Make upload_tmp a more general tmp directory
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2018-05-14 10:36:39 -03:00
d90d8c9b27 Bug 20428: Add the option to specify a tmp uploads dir
This patch adds an option to the koha-conf.xml file for specifying
a temporary uploaded files directory.

The koha-create script is adjusted to handle it and a convenient option
switch is added. If ommited, it will default to
/var/lib/koha/<instance>/uploads_tmp.

koha-create-dirs is patched to create the required directory with the
right permissions.

The docs get the new parameter documented.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Mark Tompsett <mtompset@hotmail.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2018-05-14 10:36:38 -03:00
c46439abda Bug 20612: koha-conf.xml cleanup
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Benjamin Rokseth <benjamin.rokseth@deichman.no>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2018-05-09 12:56:01 -03:00
Julian Maurice
43a4b3c22c Bug 20402: Implement OAuth2 authentication for REST API
It implements only the "client credentials" flow with no scopes
support. API clients are tied to an existing patron and have the same
permissions as the patron they are tied to.
API Clients are defined in $KOHA_CONF.

Test plan:
0. Install Net::OAuth2::AuthorizationServer 0.16
1. In $KOHA_CONF, add an <api_client> element under <config>:
     <api_client>
       <client_id>$CLIENT_ID</client_id>
       <client_secret>$CLIENT_SECRET</client_secret>
       <patron_id>X</patron_id> <!-- X is an existing borrowernumber -->
     </api_client>
2. Apply patch, run updatedatabase.pl and reload starman
3. Install Firefox extension RESTer [1]
4. In RESTer, go to "Authorization" tab and create a new OAuth2
   configuration:
   - OAuth flow: Client credentials
   - Access Token Request Method: POST
   - Access Token Request Endpoint: http://$KOHA_URL/api/v1/oauth/token
   - Access Token Request Client Authentication: Credentials in request
     body
   - Client ID: $CLIENT_ID
   - Client Secret: $CLIENT_SECRET
5. Click on the newly created configuration to generate a new token
   (which will be valid only for an hour)
6. In RESTer, set HTTP method to GET and url to
   http://$KOHA_URL/api/v1/patrons then click on SEND
   If patron X has permission 'borrowers', it should return 200 OK
   with the list of patrons
   Otherwise it should return 403 with the list of required permissions
   (Please test both cases)
7. Wait an hour (or run the following SQL query:
   UPDATE oauth_access_tokens SET expires = 0) and repeat step 6.
   You should have a 403 Forbidden status, and the token must have been
   removed from the database.
8. Create a bunch of tokens using RESTer, make some of them expires
   using the previous SQL query, and run the following command:
     misc/cronjobs/cleanup_database.pl --oauth-tokens
   Verify that expired tokens were removed, and that the others are
   still there
9. prove t/db_dependent/api/v1/oauth.t

[1] https://addons.mozilla.org/en-US/firefox/addon/rester/

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2018-05-08 15:55:42 -03:00
Roch D'Amour
67ec2ead88 Bug 11317: (follow-up) Add two-level entry to koha-conf and remove unwanted characters
- Removed merge marker
- Changed include path in favor of using the Asset tt plugin (bug 20538)
- Changed access_dir to a two-level entry for clarity

Test plans stay the same, just make sure that the two-level configuration entry
work properly and everything pass QA.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2018-05-03 13:26:50 -03:00
Roch D'Amour
0f9ec12875 Bug 11317: Add a way to access files from the intranet
This squash contains all of these commits:
- Adds a page to access log files on the server from the intranet
- Update ID to allow for permalinking
- Rename config to "'accessdir' and fix qa
- Allows for multiple directories to be accessible
- Update the link under reports
- (Follow-up) Fixing merge error and cosmetic changes
- (Follow-up) Fix tab chars and move javascript to the footer
- (QA Follow-up) Fix datatable
- Make filename unicode-proof, renamed accessdir to access_dir and fix update

Test plans:
- Apply patch, update database
- Add to koha-conf:
<access_dir>/tmp/koha-public/one</access_dir>
<access_dir>/tmp/koha-public/two</access_dir>
<access_dir>/tmp/koha-public</access_dir>
- Create these directories ( mkdir /tmp/koha-public , etc...)
- Create these files:
echo "hello world!" > /tmp/koha-public/❤
echo "test" > /tmp/koha-public/one/samename.txt
echo "this is not the same" > /tmp/koha-public/two/samename.txt
- Login as Superadmin, go to tools > reports files
    - Click on ❤, make sure it's downloadable and readable
    - Click on both samename.txt, look inside and make sure the file is different
- Login as NON-superadmin. Go under tools, see no Report/Log under the third column
    - Go to add tools/access_file permission to user
    - See new entry under tools third column.
    - validate link is ok.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2018-05-03 13:26:49 -03:00
Ere Maijala
f489d2034b Bug 20073: Move Elasticsearch configs to yaml files and improve the default settings.
Improvements:
1) Index settings moved from code to etc/searchengine/elasticsearch/index_config.yaml. An alternative can be specified in koha-conf.xml.
2) Field settings moved from code to etc/searchengine/elasticsearch/field_config.yaml. An alternative can be specified in koha-conf.xml.
3) mappings.yaml has been moved from admin/searchengine/elasticsearch to etc/searchengine/elasticsearch. An alternative can be specified in koha-conf.xml.
4) Default settings have been improved to remove punctuation from phrases used for sorting etc.
5) State variables are used for storing configuration to avoid parsing it multiple times.
6) A possibility to reset the fields too has been added to the reset operation of mappings administration.
7) mappings.yaml has been moved from admin/searchengine/elasticsearch to etc/searchengine/elasticsearch.
8) An stdno field type has been added for standard identifiers.

To test:
1) Run tests in t/Koha/SearchEngine/Elasticsearch.t
2) Clear tables search_fields and search_marc_map
3) Go to admin/searchengine/elasticsearch/mappings.pl?op=reset&i_know_what_i_am_doing=1
4) Verify that admin/searchengine/elasticsearch/mappings.pl displays the mappings properly, including ISBN and other standard number fields.
5) Index some records using the -d parameter with misc/search_tools/rebuild_elastic_search.pl to recreate the index
6) Verify that you can find the records
7) Put <elasticsearch_index_mappings>non_existent</elasticsearch_index_mappings> to koha-conf.xml
8) Verify that admin/searchengine/elasticsearch/mappings.pl?op=reset&i_know_what_i_am_doing=1 fails because it can't find non_existent.
9) Copy etc/searchengine/elasticsearch/mappings.yaml to a new location and make elasticsearch_index_mappings setting in koha-conf.xml point to it.
10) Make a change in the new mappings.yaml.
11) Clear table search_fields (mappings reset doesn't do it yet, see bug 20248)
12) Go to admin/searchengine/elasticsearch/mappings.pl?op=reset&i_know_what_i_am_doing=1
13) Verify that the changes you made are now visible in the mappings UI

Signed-off-by: Nicolas Legrand <nicolas.legrand@bulac.fr>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2018-04-23 14:22:16 -03:00
015e7ee333 Bug 20123: Make the timezone entry empty and uncommented by default
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2018-04-11 15:05:02 -03:00
Kyle M Hall
cb35784d3c Bug 20123: Add commented out example to koha-conf.xml
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2018-04-11 15:05:01 -03:00
57844add80 Bug 19739: Add default ES configuration to koha-conf.xml
Bug 18571 added default ES configuration for packaging config.

This patch adds it to dev install in koha-conf.xml.
Database name is used in index_name to allow multiple installs.

Test plan :
- Run dev install
- Install ElasticSearch server and Koha deps
- Enable ElasticSearch in Koha
- Check indexing and searching works directly

Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2018-03-27 17:35:05 -03:00
97b8bd5ebc Bug 20275: Add comment to let user know they can define multiple plugindirs
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2018-03-23 11:45:38 -03:00
dbc24783ee Bug 17826: Allow extended patron attributes to be sent in arbitrary SIP2 fields
Some libraries need to be able to send additional patron data from the
extended patron attributes in made up SIP2 fields for the patron
information and patron status responses.

Test Plan:
1) Apply this patch
2) Create 3 new patron attributes with the codes CODE1, CODE2, CODE3.
   Make a least one repeatable.
3) Create a patron, add those attibutes for the patron, make sure there
   are at least two instances of the repeatable code
4) Edit your SIP2 config file, add the following within the login stanza:
   <patron_attribute field="XX" code="CODE1" />
   <patron_attribute field="XY" code="CODE2" />
   <patron_attribute field="XZ" code="CODE3" />
5) Using the sip cli emulator, run patron_information and
   patron_status_request messages for the patron
6) Note the values you set for the patron attributes are sent in the
   corrosponding fields!

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Daniel Mauchley <dmauchley@duchesne.utah.gov>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Amended: added parentheses on line 488 when assigning hashref to array.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2018-03-23 11:45:38 -03:00
818f1c2f83 Bug 20187: (QA follow-up) Replace N,L flag by L flag
The flags [N,L] make no sense: next and last combined.
Choosing here for L to stop the rewriting process.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Victor Grousset <victor.grousset@biblibre.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2018-02-13 12:54:32 -03:00
8cd0a76d10 Bug 20187: New rewrite rules can break custom css
Koha has the ability to include custom css in the apache configuration.
If a library has any custom css ( or adds a custom js file in some way ),
and that file has an underscore in it ( e.g. my_custom.css ), the
apache rewrite rule will convert it to my.css and thus it will 404.

We should make the rewrite rules as specific as possible for the
format we are using.

Test Plan:
1) Set OPAC_CSS_OVERRIDE to a file with an underscore in it
2) Note it does not work
3) Apply this patch
4) Update the apache rewrite rules to match those in the patch
   For kohadevbox, just run /home/vagrant/misc4dev/cp_debian_files.pl
5) Restart apache
6) Reload the page, your custom css should load now!

Signed-off-by: Victor Grousset <victor.grousset@biblibre.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2018-02-13 12:54:32 -03:00
Olli-Antti Kivilahti
f89a2bbdbe Bug 12904: Add mod_expire to etc/koha-httpd.conf
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
2018-02-08 14:53:23 -03:00