Commit graph

691 commits

Author SHA1 Message Date
af1f8d34db Bug 30728: Only trigger real-time holds queue update if enabled
This patch makes the places in which Koha enqueues holds queue for
real time updates verify the feature is enabled.

To test:
1. Apply this patches up to the unit tests
2. Run:
   $ updatedatabase
   $ kshell
  k$ git diff origin/master --name-only | grep -e '\.t$' | xargs prove
=> FAIL: tests fail, the code doesn't care about the syspref
3. Apply this patch
4. Repeat 2
=> SUCCESS: Tests pass!
5. Be happy!
6. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-05-12 22:17:46 -10:00
63002fa589 Bug 30710: (QA follow-up) Add POD
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-05-10 23:09:09 -10:00
a3649905df Bug 30710: Build holds queue once per biblio when batch deleting items
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-05-10 23:09:09 -10:00
Roch D'Amour
55d1e68024 Bug 13412: Allow configuration of auto-created authorities
This patch add 2 new system preferences:
- GenerateAuthorityField667
- GenerateAuthorityField670

When Koha creates authority records from biblios, use these preferences instead of the hard-coded "Machine generated authority record." and "Work cat." values.

Test plan:
    1. Make sure AutoCreateAuthorities and BiblioAddsAuthorities are enabled in the system preferences
    2. Go to Home › Cataloging › Add MARC record and add a new record
    3. Fill in the required fields and add a name in the 100$a field. Note the name, and add the new record.
    4. run rebuild_zebra.pl -a
    5. Go to Home › Authorities and search for the name you entered in 100$a
    6. Select details and look at the 667 and 670 fields. They should default to the hard-coded values.
    7. Apply patch, run update database
    8. Edit the new GenerateAuthorityField667 and GenerateAuthorityField670 system preferences
    9. Do 2-6 . Now the 667 and 670 fields should be what you entered in the system preferences.

Thanks for testing

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Works well, no errors

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-05-10 23:09:09 -10:00
734a02cf82 Bug 30465: Make BatchUpdateBiblio update the index in one request
When using the batch record modification tool to modify several
bibliographic records, we don't want to send one index request per
biblio, we want to index them all on the fly after the records have been
modified.
Otherwise we will end up with one task per record, and records will be
indexed in background.

Test plan:
Use the batch mod tool to modify bibliographic records and confirm the
behaviour is correct.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-05-05 11:17:37 -10:00
763d73fbf0 Bug 29346: Add more fine-grained control of holds queue updates
This patch deals with the fact that high-level circualtion methods like
`AddIssue`, `AddReturn` and `ModDateLastSeen` all eventually call
lower-level methods like ModBiblio, Koha::Item->store of
UpdateTotalIssues which are expected to trigger holds queue updates (for
the object CRUD operations use cases). As the circulation methods need
to trigger holds queue update as well, duplicate updates were being
requested which is suboptimal, of course.

In order to prevent this, and because circulation methdos could trigger
holds queue updates several times, actually, I added a new parameter
*skip_holds_queue* to the low-level methods, so when they are called
from circulation, the trigger is skipped and we have greater control on
when and how holds queue updates are scheduled.

This patch introduces the `skip_holds_queue` parameter to the following
methods:

* C4::Biblio::ModBiblio
* C4::Biblio::UpdateTotalIssues
* Koha::Item->store

Calls to those methods from the following methods will include the new
parameter, and thus duplicated holds queue updates avoided:

* C4::Circulation::AddIssue
* C4::Circulation::AddReturn
* C4::Items::ModDateLastSeen

Tests are added, to verify that the (mocked) BatchUpdateBiblioHoldsQueue
task is only scheduled once when they are called.

To test:
1. Apply up to the previous patch
2. Run:
   $ kshell
  k$ prove t/db_dependent/Biblio.t \
           t/db_dependent/Biblio_holdsqueue.t \
           t/db_dependent/Circulation_holdsqueue.t
=> FAIL: Tests fail!
3. Apply this patch
4. Repeat 2
=> SUCCESS: Tests pass!
5. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-05-05 11:17:36 -10:00
a6d501ad8a Bug 29346: Biblio actions triggers
This patch adds the trigger for the holds queue update on teh following
methods:

- C4::Biblio::DelBiblio
- C4::Biblio::ModBiblio

The ModBiblio use case could be improved by checking if itemtype is one
of the updated attributes... but it felt there was no way to do it
without some overhead. So I leave it as-is.

It also mocks the ->enqueue method in the tests that call DelBiblio and
ModBiblio to avoid breakages [1]

Tests are added to check the trigger is called.

To test:
1. Apply this patch
2. Run:
   $ kshell
  k$ prove t/db_dependent/Biblio.t
=> SUCCESS: Tests pass! It means the trigger is triggered :-D
3. Sign off :-D

[1] This breakages could be avoided if we solve Koha/BackgrounJob.pm:101
FIXME.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-05-05 11:17:35 -10:00
3c2e92448a Bug 29486: _koha_marc_update_bib_ids no longer needed for GetMarcBiblio
This subroutine is ensuring that the biblionumber and biblioitemnumber
will be part of the MARC record.
We should not need that, unless there is something broken somewhere
else.

This line has been added by the following commit:
  commit 4e95e94727
  Bug 6789: biblios with many items can result in broken search results link

"""
To this end, it also moves the fix_biblio_ids portion of get_corrected_marc_record out of rebuild_zebra.pl,
and makes it a part of GetMarcBiblio (right before EmbedItemsInMarcBiblio, so the 952s still come last).  fix_biblio_ids
is kept as a subroutine for the deletion portion of rebuild_zebra.pl, which still uses it.
"""

But it does not explain why it's better to have it in GetMarcBiblio.
If we need it for the reindexation process, we shouldn't impact
GetMarcBiblio which is used from several different places.

We might then consider adding the fix_biblio_ids call to
rebuild_zebra.pl, but I am failing to understand in which cases it could
be useful.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-03-22 10:17:34 -10:00
David Gustafsson
7da4912f8e Bug 25616: Uppercase hard coded lower case boolean operators for Elasticsearch
Uppercase occurances of all (hopefully) lowercase "and"
used in ElasticSearch Query String Query contexts

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-02-24 14:35:36 -10:00
Aleisha Amohia
d5621ce951 Bug 27266: Move GetMarcAuthors to Koha namespace
This patch moves C4::Biblio::GetMarcAuthors to
Koha::Biblio->get_authors_from_MARC. This is so the method can be
used in templates and notices.

To test:
1. Find a record that has an author in the added entry field (700-720).
2. Add the record to the cart and a list.
3. View your cart and click 'more details'. Confirm authors show as
normal.
4. Click 'send' and confirm the email sent shows the authors as normal.
5. Go to the list you added the record to and click 'send list'. Confirm
the email sent shows the authors as normal.
xslt)
6. Log in to the OPAC. Find the record and add it to the cart and a list
7. View the cart and click 'more details'. Confirm authors show as
normal.
8. Click 'send' and confirm the email sent shows the authors as normal.
9. Go to the list you added the record to and click 'send list'.
Confirm the email sent shows the authors as normal.
10. Confirm tests pass:
- t/Biblio.t
- t/db_dependent/Koha/Biblio.t

Sponsored-by: Bibliotheksservice-Zentrum Baden-Württemberg (BSZ)
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2021-12-15 12:15:03 -10:00
ada1c03350 Bug 24674: (follow-up) Simpler regex
The 'Polish notation' actually simplifies things.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-11-05 12:17:08 +01:00
Janusz Kaczmarek
5fa32f2394 Bug 24674: Uncertain years for publicationyear/copyrightdate -- corrected
This is a correction to Bug 11046: YYY-, YY--, Y---, are also
correct according to ISBD: uncertain year, but certain decade/century
(cf. n. 4.3.8 in ISBD Consolidated Edition of 2011).
(Question mark is not mandatory.)

In addition, in Poland a form with one only dash is quite common,
like YY- (== YY--).

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
See last follow-up.

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-11-05 12:17:08 +01:00
27f3d0af63 Bug 14957: (QA follow-up) Clarify 'context' param
This patch renames the (passed through) 'context' param for
'overlay_context'. I propose doing so, because in Koha-land 'context'
has a special meaning, related to C4::Context and it reads ambigous.

The patch itself is pretty trivial.

Tests should pass:
1. Run:
   $ kshell
  k$ prove t/db_dependent/Biblio/MarcOverlayRules.t
=> SUCCESS: Tests pass
2. Apply this patch
3. Repeat 1
=> SUCCESS: Tests still pass!
4. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Bug 14957: (follow-up) Clarify 'context' param

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-26 16:46:02 +02:00
02720d423c Bug 14957: (QA follow-up) Rename 'merge' => 'overlay'
This patch acknowledges the fact that 'merging' has a different meaning
in Koha than the behavior this great patchset introduces. The more
idiomatic way of describing the behavior is to talk about 'overlay rules'.

This patch also:
- Fixes kohastructure.sql missing table encoding a collation
- Fixes the atomic update completely missing a table
- Moves the syspref entry to 'Importing', probably a rebase issue

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Bug 14957: (QA follow-up) Fix syntax error in atomicupdate

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Bug 14957: Fix imports in tests

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Bug 14957: Fix imports in marc-overlay-rules.pl

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Bug 14957: Fix syspref's values

It didn't switch off when set to "don't use"

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

JD amended patch: Fix license statement

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-26 16:46:02 +02:00
David Gustafsson
139e6c30d6 Bug 14957: Merge rules system for merging of MARC records
Add a rule based system for merging MARC records to for example
prevent field data from being overwritten.

To test:
    1. Apply this patch.
    2. Log in to staff client.
    3. Enable new syspref MARCMergeRules.
    4. Click the new link "MARC merge rules" in the "Catalog"
       section of the Koha administration page.
    5. Create a new rule:
       Module: source, Filter: *, Tag: 245, Preset: Protect.
    6. Clicking "Edit" should allow you to edit corresponding rule.
    7. Clicking "Delete" should remove corresponding rule after confirmation.
    8. Selecting one or more rules followed by clicking "Delete
       selected" should remove all selected rules after confirmation.
    9. Try creating a rule with tag set to "**", the other options does
       not matter. Verify that saving this rule produces an error
       message complaining about invalid tag regular expression.
    10. Try creating a rule with tag set to "008" (or other control
        field) and set Appended: Append and Removed: Skip, the other
        options does not matter. Verify that saving this rule produces
        an error message complaining about invalid combination of actions
        for control field.
    11. With the 245 rule in step 5 in place, edit a bibliographic record,
        change 245a for example (which should be Title for MARC21) and save.
    12. Verify that the changes has not been saved.
    13. Create a new rule:
        Module: source, Filter: intranet, Tag: 245, Preset: Overwrite.
    14. Repeat step 12, and verify that the changes has now been saved.
    15. Run tests in t/db_dependent/Biblio/MarcMergeRules.t and very
        that all tests pass.

Sponsored-by: Halland County Library
Sponsored-by: Catalyst IT
Sponsored-by: Gothenburg University Library
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Christian Stelzenmüller <christian.stelzenmueller@bsz-bw.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-26 16:46:02 +02:00
08c03af2ae Bug 18984: Remove NORMARC support
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-07 15:36:40 +02:00
6560049c65 Bug 27526: Fix cn_source display
For an unknown reason C4::Biblio::GetAuthorisedValueDesc (that we are
calling from Koha::Item->columns_to_str) does not deal with class
sources.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-06 11:44:22 +02:00
96826fd2d0 Bug 27526: Remove ModItemFromMarc from additem
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-06 11:44:22 +02:00
9d6d641d1f Bug 17600: Standardize our EXPORT_OK
On bug 17591 we discovered that there was something weird going on with
the way we export and use subroutines/modules.
This patch tries to standardize our EXPORT to use EXPORT_OK only.

That way we will need to explicitely define the subroutine we want to
use from a module.

This patch is a squashed version of:
Bug 17600: After export.pl
Bug 17600: After perlimport
Bug 17600: Manual changes
Bug 17600: Other manual changes after second perlimports run
Bug 17600: Fix tests

And a lot of other manual changes.

export.pl is a dirty script that can be found on bug 17600.

"perlimport" is:
git clone https://github.com/oalders/App-perlimports.git
cd App-perlimports/
cpanm --installdeps .
export PERL5LIB="$PERL5LIB:/kohadevbox/koha/App-perlimports/lib"
find . \( -name "*.pl" -o -name "*.pm" \) -exec perl App-perlimports/script/perlimports --inplace-edit --no-preserve-unused --filename {} \;

The ideas of this patch are to:
* use EXPORT_OK instead of EXPORT
* perltidy the EXPORT_OK list
* remove '&' before the subroutine names
* remove some uneeded use statements
* explicitely import the subroutines we need within the controllers or
modules

Note that the private subroutines (starting with _) should not be
exported (and not used from outside of the module except from tests).

EXPORT vs EXPORT_OK (from
https://www.thegeekstuff.com/2010/06/perl-exporter-examples/)
"""
Export allows to export the functions and variables of modules to user’s namespace using the standard import method. This way, we don’t need to create the objects for the modules to access it’s members.

@EXPORT and @EXPORT_OK are the two main variables used during export operation.

@EXPORT contains list of symbols (subroutines and variables) of the module to be exported into the caller namespace.

@EXPORT_OK does export of symbols on demand basis.
"""

If this patch caused a conflict with a patch you wrote prior to its
push:
* Make sure you are not reintroducing a "use" statement that has been
removed
* "$subroutine" is not exported by the C4::$MODULE module
means that you need to add the subroutine to the @EXPORT_OK list
* Bareword "$subroutine" not allowed while "strict subs"
means that you didn't imported the subroutine from the module:
  - use $MODULE qw( $subroutine list );
You can also use the fully qualified namespace: C4::$MODULE::$subroutine

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-07-16 08:58:47 +02:00
5da5a2bbc5 Bug 28384: Add 'no_items' option to TransformMarcToKoha
This patch adds a new option 'no_items' and uses it in the C4::Search::searchResults
routine. We don't use the item info fetched here, so skipping those lines saves us time.

Additionally, I fix an incorrect FIXME comment, and pass the ISBN returned by the routine above
into GetNormalizedISBN to save another lookup

TO test:
1 - Enable AmazonCoverImages system preference
2 - Search staff client with a term that returns books with covers
3 - Apply patch
4 - prove -v t/db_dependent/Biblio/TransformMarcToKoha.t
5 - Confirm searching works
6 - Confirm Amazon images display (normalized_isbn is used for these)

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] Amended, replacing a tab character.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-07-12 15:47:34 +02:00
3fe837bc74 Bug 28572: Remove C4::Debug
We are using Koha::Logger when it makes sense to keep the info,
otherwise we simply remove it

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Bug 28572: Replace missing occurrence in misc/admin/koha-preferences

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-06-22 12:04:32 +02:00
f5bd16410a Bug 27837: Always pass permanent location from items editor if it was defined
By always passing permenent location from the items editor, we can know
if permanent location was defined in the framework. This means we can
know if we should update permanent location independently of location
even if permenent location has an empty value.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-05-19 15:06:42 +02:00
b3baca4137 Bug 27577: Autolink biblio after biblionumber is generated
If BiblioAddsAuthorities and AutoCreateAuthorities are enabled, a new record is auto linked when saving and authorities generated with a 670 field that indicates which record generated the new authority.

The code to do this is called before the biblionumber is generated, so the variable that adds the biblionumber to the citation is empty

This patch moves the linking code to after the biblionumber is generated

To recreate:
1 - Enable BiblioAddsAuthorities and AutoCreateAuthorities
2 - Import or create a new record
3 - Ensure the record has controlled fields that do not have authorities before saving, i.e.:
    Add a 650 with "test_bug_27577_01"
4 - Search for the authority record
5 - Note the 670 field does not contain the biblionumber
6 - Apply patch, restart all the things
7 - Repeat 2-4
8 - Note the 670 now has the biblionumber

Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-05-06 15:52:38 +02:00
Ere Maijala
016bb29b60 Bug 21958: Fix bibliographic record field comparison with authority
This makes the comparison between bibliographic field and authority field more robust and per subfield. This makes the comparison not consider the same e.g. the following fields:

$a Test User
$a Test $b User

The actual issue cannot be as easily reproduced with the patches for bug 21826 applied, but here's a test plan anyway:

1. Make sure tests pass (especially t/db_dependent/AuthoritiesMarc*)
2. Make sure authority linking still works properly
3. Make sure authority and biblio frameworks allow subfield i
4. Make sure that even if you add subfield i to 700 in biblio, authority link is kept the same
5. Make sure that even if you add subfield i to the authority record, the authority link is kept the same

Signed-off-by: Frank Hansen <frank.hansen@ub.lu.se>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-04-01 17:13:56 +02:00
Aleisha Amohia
90491bb85a Bug 27268: Move GetMarcNotes to Koha namespace
This patch moves C4::Biblio::GetMarcNotes to
Koha::Biblio->get_marc_notes. This is so get_marc_notes can be
used in templates and notices.

To test:
1. Find a record that has a note (3xx for UNIMARC, 5xx for MARC21).
Confirm the notes still show as normal under the Descriptions tab.
2. Add the record to the cart.
3. View your cart and click 'more details'. Confirm notes show as
normal.
4. Log in to the OPAC. Find the record and add it to the cart
5. View the cart and click 'more details'. Confirm notes show as
normal.
6. View the record detail page and confirm the notes show as normal
under the Title Notes tab.
7. Confirm tests pass:
- t/Biblio.t
- t/db_dependent/Koha/Biblio.t

Sponsored-by: Bibliotheksservice-Zentrum Baden-Württemberg (BSZ)

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-03-09 17:20:47 +01:00
8650f8111b Bug 8976: Code cleaning
It was tricky to write the previous patch so I decided to clean the
ground (a bit) for later, taking advantage of having people who are
going to test the whole area.
We can move it to its own bug report if QA wants to.

Sponsored-by: Orex Digital

Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-03-01 15:14:22 +01:00
b387502eaf Bug 8976: Add the ability to sort subfields for the MARC frameworks
This new enhancement adds the ability to change the default order the subfields.

Both bibliographic and authority MARC subfield structure are taken into
account. And so the item edition as well (in the different screens when
item can be added/edited).

This will answer the following needs that have been listed in the
comments of the bug report:
- $i in 7xx fields should be the first subfield in the sequence
- 300 fields are sorted number first when cataloguers enter the letter fields first
- 100 field, it's commonly $a, $q, $d.

Test plan:
1. Edit a MARC frameworks, field 300
2. Resort the subfield (drag and drop the tab of the subfield) as you
like
3. Save
=> Notice that the list of fields are displayed following the order you
chose
4. Edit it again
=> The order is correctly kept!
5. Create a new bibliographic record
6. Notice that the subfields are order in the same sequence
7. Fill different subfields, not all
8. Save, edit again
9. Note that the subfields that have been filled are listed first, then
the empty ones. But the sequence defined at the framework level is kept.
10. Do the same for an authority framework and create/edit an authority
record
11. Modify item (952) subfields order
12. Create an item and confirm that the order is correct
13. Modify the ACQ framework, 952, modify the order of the subfield
14. Create a new order and confirm that the item form has the subfield
ordered following the sequence defined at the framework level

QA: Note that this patch is about bibliographic records only, next
patches deal with authotiries and items.

Sponsored-by: Orex Digital

Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-03-01 15:14:22 +01:00
bf7a645cbe Bug 11299: (follow-up) Rebase on recent changes
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-01-14 14:03:49 +01:00
85950bdbc1 Bug 11299: (QA follow-up) Fix tests, spaces, and CSS update
Removed my from variables in test

Undid stray line deletions

Undid adding 1; to test file

Fix CSS replace command

Update license

Fix spelling

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-01-14 14:03:49 +01:00
Bouzid Fergani
11de340639 Bug 11299: Add a button to the biblio edition page to automatically add authority links in the current biblio record via AJAX. Also adds a button to easily create missing authority records.
add authority type in the form to create the missing  authority.
 when authority was found, the 600$9 field have the authid.

Testing scenario (Creating an authority record for a failed automatic link) :

 1 - In your system preferences set:
    AutoCreateAuthorities: Don't generate
    BiblioAddsAuthorities: Allow
 2 - Go to the Cataloging -> New record (koha/cataloguing/addbiblio.pl)
    Ensure you are using the basic editor
 3 - Click the "Link authorities automatically" button.
    A message should appear, telling the user "No authority link was changed."
 4 - Add random informations in field 600$a of the biblio record.
 5 - Click the "Link authorities automatically" button.
   the message box should now show  "600 - No matching authority found.".
   the 9 subfield is red
   Above the 9 subfield is a red X with a blue plus next to it
   Hover on the plus, see it is titled 'Create authority'
 6 - Click the 'Create authority' link
 7 - A new authroity form pops up, the info from the cataloging editor is prefilled
     Click the 100 field heading to expand and confirm info is transferred
 8 - Fill in necessary fields and save the new authority
 9 - The cataloging screen now has the 9 subfield populated and is green
10 - Click "Link authorities automatically" again
     Dialog says "No authority link was changed"
11 - In another tab go to System preferences and set AutoCreateAuthorities to 'Generate'
12 - Add random information to the 650 field
13 - Click 'Link authorities' button
14 - Dialog says:650 - No matching authority found. A new authority was created automatically.
15 - The subfield 9 is green and has the id of the new authority record
16 - In another tab search authorities and find an existing subject heading
17 - Add a new 650 with the info from the existing record
18 - Click 'Link authorities'
19 - The new field is correctly linked to existing authority

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

https://bugs.koha-community.org/show_bug.cgi?id=12299
Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Michal Denar <black23@gmail.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-01-14 14:03:49 +01:00
a96c73eb76 Bug 25306: Remove framework paramter from ModBiblioMarc
Tested with :

For a framework (not the default) :

Creation of a biblio record
Edition of this biblio record
Creation of an item of this record
Creation of an item of this record

./misc/batchRepairMissingBiblionumbers.pl OK

prove t/db_dependent/Biblio/ModBiblioMarc.t OK

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-01-12 16:13:50 +01:00
9038cdd9a4 Bug 26518: Move BiblioAutoLink within the transaction
We don't want the authorities to be created if the biblio insert fails
later

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-11-25 15:18:17 +01:00
b0c39296c5 Bug 26518: Raise exception if the insert failed
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-11-25 15:18:17 +01:00
a1e9453f1d Bug 26518: Use Koha::Biblio[item] in AddBiblio
Bug 26518: Does not return from the catch block

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-11-25 15:18:17 +01:00
e1400890ec Bug 26516: Don't accept incorrect values for copyrightdate/publicationyear
The previous patch rejects incorrect values when saving to the db specifically for copyrightdate
An error is thrown for the int conversion when it fails

Rather than catching things when saving we can fix the value when generated, simply returning null when parsing
the record

To test:
1 - Add a new record to Koha making sure data is valid except 260$c:
    198-
2 - Save the record
3 - It fails
4 - Apply this patch
5 - Restart all the thigns
6 - Repeat
7 - Success!

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-11-16 14:25:21 +01:00
Andreas Roussos
ab2e3b0e89 Bug 26641: pass the MARC field as an argument to link_bibs_to_authorities.pl
When running link_bibs_to_authorities.pl, it could be useful to have
the ability to specify which MARC field the script should operate on.

For example, when you have a catalog where most links are in place,
and you don't want the script to process every MARC field but rather
limit it to a specific MARC field (e.g. 700) because you've observed
that there are many missing links for that particular MARC field.

This patch provides that enhancement.

Test plan:
1) Run the script as follows (preferably in a test DB):

   time link_bibs_to_authorities.pl -v -l --test

   ... and notice how long it takes to complete.

2) Apply the patch.

3) Run the script again, this time providing the MARC field to work on:

   time link_bibs_to_authorities.pl -v -l --test -g=700

   ... and, again, notice how long it takes to complete.

   The 2nd run should run faster than the 1st one.

Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-11-04 12:59:34 +01:00
0113df1a7a Bug 26599: Unused parameter name in POD of ModZebra
This patch removes the $record reference in POD.
Nothing else ;)

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-10-08 14:50:06 +02:00
501b9ea336 Bug 25265: (QA follow-up) Rename biblionumber in ModZebra, index_records
ModZebra:
The name is very misleading: we can index authid's too here.
And yes, it should not be in C4/Biblio too ;) A first step..

Adding the same change here in Koha/SearchEngine/Zebra/Indexer.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-09-28 10:10:08 +02:00
88cb7f223d Bug 25265: Prevent double reindex of the same item in batchmod
When batch editing, 2 reindex calls are sent to ES/Zebra.
We can easily avoid that reusing the skip_modzebra_update (renamed skip_record_index)

Additionally we should only send one request for biblio, and we should
only do it if we succeed

As the whole batch mod is in a transaction it is possible to fail in which case
Zebra queue is reset, but ES indexes have already been set

In addition to the skip param this patchset moves Zebra and Elasticsearch calls to
Indexer modules and introduces a generic Koha::SearchEngine::Indexer so that we don't
need to check the engine when calling for index

The new index_records routine takes an array so that we can reduce the calls to
the ES server.

The index_records routine for Zebra loops over ModZebra to avoid affecting current behaviour

Test plan:

General tests, under both search engines:
  1 - Add a biblio and confirm it is searchable
  2 - Edit the biblio and confirm changes are searchable
  3 - Add an item, confirm it is searchable
  4 - Delete an item, confirm it is not searchable
  5 - Delete a biblio, confirm it is not searchable
  6 - Add an authority and confirm it is searchable
  7 - Delete an authority and confirm it is not searchable

Batch mod tests, under both search engines
  1 - Have a bib with several items, none marked 'not for loan'
  2 - Do a staff search that returns this biblio
  3 - Items show as available
  4 - Click on title to go to details page
  5 - Edit->Item in a batch
  6 - Set the not for loan status for all items
  7 - Repeat your search
  8 - Items show as not for loan
  9 - Test batch deleting items
    a - Test with a list of items, not deleting bibs
    b - Test with a list of items, deleting bibs if no items remain where all items are only item on a biblio:
     SELECT MAX(barcode) FROM items GROUP BY biblionumber HAVING COUNT(barcode) IN (1)
    c - Test with a list of items, deleting bibs if no items remain where some items are the only item on a biblio:
     SELECT MAX(barcode) FROM items GROUP BY biblionumber HAVING COUNT(barcode) IN (1,2)
 10 - Confirm records are update/deleted as appropriate

Signed-off-by: Bob Bennhoff <bbennhoff@clicweb.org>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-09-28 10:10:08 +02:00
ce161fda9b Bug 25273: Make match-heading rely on authority type configuration
The match-heading field is a special field used only by the linker, not accessible
to staff or patrons via the interface. This field is used to store the constructed
'search form' used for matching bib headings to authority fields.

In bug 24269 I attempted to use the mappings defined in the inferface and also inject the search term.
This did not work as too many subfields were indexed on their own and leading to false matches.
In this bug we remove the mappings for this field, and create it ourselves during
the indexing process. The C4::Headings module is still used to generate the correct form,
however, the mappings are set based on the authority types in the system. This gives the user
the ability to add new typoes, but prevents mapping changes from breaking linker functionality

To test:
 1 - Start form a sample database with ElasticSearch working
 2 - Download via Z39.50 2 authorities, one of which is a narrower heading of the other, e.g.:
    Waterworks
    Waterworks - Costs
 3 - Place a heading for the broader term in a record. e.g. Waterworks
       In 650$a, without the cataloguing authority plugin. We don't want
       the link created now.
       You need syspref BiblioAddsAuthorities => allow
 4 - Make sure linker is set to default
 5 - Attempt to link the records
       misc/link_bibs_to_authorities.pl
 6 - Linking fails
 7 - Apply patch
 8 - refresh index settings (if using a custom file, remove 'match-heading')
       You can reset mappings in the UI or run this:
       misc/search_tools/rebuild_elasticsearch.pl -v -d -r
 9 - Reindex ES
10 - Try to link again
11 - It succeeds!
12 - Run the tests
     prove t/db_dependent/Koha/SearchEngine/Elasticsearch.t

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Bug 25273: (follow-up)

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-08-31 16:10:25 +02:00
88ecaa54cd Bug 26133: Remove GetMarcHosts
We removed the single occurrence of this subroutine in the previous
patch

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-08-05 17:36:28 +02:00
50302c714e Bug 26063: Use Koha::Plugins->call for some other hooks
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-07-30 17:44:23 +02:00
5e45076c00 Bug 25709: Rename systempreference to NotesToHide
In an effort to use more inclusive language we have added a new coding guideline:
https://wiki.koha-community.org/wiki/Coding_Guidelines#TERM3:_Inclusive_Language

This patchset renames a syspref to be clearer and follow the guideline

To test:
1 - Apply patch and updatedatabase
2 - git grep NotesBlacklist
3 - Note all remaining occurrences are translations, db updates, and release notes
4 - Add a field (e.g. '520' ) to the NotesToHide systempreference
5 - View a record with a 520 field on the opac
6 - Confirm the field does not show in the 'Title notes' tab

Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-07-29 10:08:43 +02:00
Julian Maurice
b168f4a2e9 Bug 21395: Make perlcritic happy
This patch adds a .perlcriticrc (copied from qa-test-tools) and fixes
almost all perlcrictic violations according to this .perlcriticrc
The remaining violations are silenced out by appending a '## no critic'
to the offending lines. They can still be seen by using the --force
option of perlcritic
This patch also modify t/00-testcritic.t to check all Perl files using
the new .perlcriticrc.
I'm not sure if this test script is still useful as it is now equivalent
to `perlcritic --quiet .` and it looks like it is much slower
(approximatively 5 times slower on my machine)

Test plan:
1. Run `perlcritic --quiet .` from the root directory. It should output
   nothing
2. Run `perlcritic --quiet --force .`. It should output 7 errors (6
   StringyEval, 1 BarewordFileHandles)
3. Run `TEST_QA=1 prove t/00-testcritic.t`
4. Read the patch. Check that all changes make sense and do not
   introduce undesired behaviour

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-06-29 12:37:02 +02:00
980419ed15 Bug 25189: Don't create authority if results found
Automatic authority creation assumes that if we don't match we need a new authority.

Using the Default linker, however, we don't match if there exists more than one match.
This leads to repeatedly generating authorities once there is a duplicate in the system

We shoudl instead only create a new authority if there are no results

To test:
1 - Set Linker Module to 'Default'
2 - Enable  AutoCreateAuthorities  and  BiblioAddsAuthorities and  CatalogModuleRelink and LinkerRelink
3 - Add two copies of a single authority via Z39
4 - Add a heading for that authority to a bib record
5 - Save the record and note a new authority is generated
6 - Repeat and see another is generated
7 - Apply patch
8 - Restart all the things
9 - Save the record again, no new authority created

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-06-24 14:31:37 +02:00
9287585f38
Bug 21901: Add FK on subscription and serial tables
In order to improve performance in the serial modules and add DB constraints,
this patch is going to add missing foreign key on the following columns:
 * serial.biblionumber
 * serial.subscription
 * subscriptionhistory.biblionumber
 * subscriptionhistory.subscriptionid
 * subscription.biblionumber

Once done, some code can be removed from the Del* subroutines, as the ON
CASCASE clause will make the RDBMS handles the deletions.

Test plan:
0/ It would be useful to test the update DB entry on a big and old
production DB, to make sure the constraints will be added correctly.
We could remove the entries before creating the constraints, but it can
be unecessary
1/ Make sure you can recreate a fresh install with the kohastructure.sql
from this patch
2/ Make sure you can upgrade from a master install
3/ Create a subscription, serial, etc. and delete the biblio
=> The subscription/serials should have been deleted from the DB
4/ Create a subscription, serial, etc. and delete the subscription
=> The serials should have been deleted from the DB

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-04-17 09:39:00 +01:00
1be518588b
Bug 24715: Cache repeatable subfield in TransformKohaToMarc
Implemented by calling GetMarcSubfieldStructure and improving
_check_split which may check repeatability in a specific framework,
and return the framework hash.

Test plan:
Run t/db_dependent/Biblio/TransformKohaToMarc.t
Optionally, follow the test plan of bug 21800#comment7.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Ere Maijala <ere.maijala@helsinki.fi>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-04-17 09:25:06 +01:00
21c9b685bf
Bug 20415: Remove UseKohaPlugins system preference
Owen Leonard 2018-03-16 10:47:47 UTC :
<<
I don't think the system preference adds any security. There are already multiple permissions required for working with plugins:

- Configure plugins
- Manage plugins ( install / uninstall )
- Use report plugins
- Use tool plugins

And even with those permissions your server must be configured to allow the use of plugins.
>>

Test plan :
1) Install kitchen sink plugin https://github.com/bywatersolutions/koha-plugin-kitchen-sink
2) Run misc/devel/install_plugins.pl
3) Set config enable_plugins=1
4) Check all parts of the plugin are working
5) Set config enable_plugins=0
6) Check all parts of the plugin are disabled

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-26 11:42:02 +00:00
01219b09ba
Bug 21800: Check the correct framework for the repeatable tag
This depends on the framework parameter. Which should be added back to
the call in C4::Items.

Test plan:
[1] Mark in Default framework one subfield A repeatable and B not repeatable.
Go to item editor. (Work on a biblio in Default framework.)
Check saving and reopening these subfields with VAL1 | VAL2.
Subfield A should be cloned, B should be glued as entered.

[2] Mark in another Framework A not repeatable and B repeatable.
Change framework for this biblio.
Go to item editor again. Reopen item. Behavior subfields in reverse?

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Ere Maijala <ere.maijala@helsinki.fi>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-24 10:54:46 +00:00
f952067cb2
Bug 21800: Make TransformKohaToMarc aware of non-repeatable subfields
If a kohafield (in Koha to MARC mappings) contains a pipe char (say A | B),
we split it up into two subfields A and B in MARC.
We will only do that for repeatable subfields now. If the field is not
repeatable, the value will just be 'A | B'.

Note 1: As bug 10306 and its friends (19096) made the Default framework
authoritative, we do no longer have the frameworkcode in this routine.
Formally, we should check the corresponding framework.

Note 2: Does this impact the reverse operation in TransformMarcToKoha?
No, the check on repeatable subfields is done in the interface and not
in TransformMarcToKoha. This routine simply translates two instances of the
same subfield, say A and B, into the value 'A | B' for a kohafield. Not
allowing two instances of a non-repeatable subfield is not in the scope of
this report.

Test plan:
[1] Mark an item field as repeatable in the Default framework.
    Edit an item. Insert A|B in this field and another not-repeatable
    field. Save and reopen. Verify that the repeatable field is duplicated
    and the other one contains the pipe character in the text box.
[2] Look for a repeatable subfield in MARC like e.g. 260$c.
    Go to the cataloguing editor and add A|B in this field.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Ere Maijala <ere.maijala@helsinki.fi>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-03-24 10:54:46 +00:00