Test plan:
(A) Reproduce the bug:
A1 In Administration > System preferences > Web services:
- Enable OAI-PMH
- Enable OAI-PMH:AutoUpdateSets
- Enable OAI-PMH:AutoUpdateSetsEmbedItemData
A2 In Tools > Export data > Export bibliographic records, export 1 (one)
biblio record. Be sure that this biblio record has at least one item.
Don't tick "Don't export items".
A3 Delete the exported biblio record, and its items.
A4 In command line, on the server load the exported file:
./bulkmarcimport.pl -b -v -file /path/to/koha.mrc
A5 Retrieve the biblio record in Koha. Note the absence of the item(s). This is
the bug.
(B) Apply the patch
B1 Delete the record (without item) loaded at A4.
B2 In command line, on the server load the exported file:
./bulkmarcimport.pl -b -v -file /path/to/koha.mrc
B2 Retrieve the biblio record in Koha. Note the presence of the item(s).
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Includes:
Bug 29697: (follow-up) Use flag embed_items
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
On bug 17591 we discovered that there was something weird going on with
the way we export and use subroutines/modules.
This patch tries to standardize our EXPORT to use EXPORT_OK only.
That way we will need to explicitely define the subroutine we want to
use from a module.
This patch is a squashed version of:
Bug 17600: After export.pl
Bug 17600: After perlimport
Bug 17600: Manual changes
Bug 17600: Other manual changes after second perlimports run
Bug 17600: Fix tests
And a lot of other manual changes.
export.pl is a dirty script that can be found on bug 17600.
"perlimport" is:
git clone https://github.com/oalders/App-perlimports.git
cd App-perlimports/
cpanm --installdeps .
export PERL5LIB="$PERL5LIB:/kohadevbox/koha/App-perlimports/lib"
find . \( -name "*.pl" -o -name "*.pm" \) -exec perl App-perlimports/script/perlimports --inplace-edit --no-preserve-unused --filename {} \;
The ideas of this patch are to:
* use EXPORT_OK instead of EXPORT
* perltidy the EXPORT_OK list
* remove '&' before the subroutine names
* remove some uneeded use statements
* explicitely import the subroutines we need within the controllers or
modules
Note that the private subroutines (starting with _) should not be
exported (and not used from outside of the module except from tests).
EXPORT vs EXPORT_OK (from
https://www.thegeekstuff.com/2010/06/perl-exporter-examples/)
"""
Export allows to export the functions and variables of modules to user’s namespace using the standard import method. This way, we don’t need to create the objects for the modules to access it’s members.
@EXPORT and @EXPORT_OK are the two main variables used during export operation.
@EXPORT contains list of symbols (subroutines and variables) of the module to be exported into the caller namespace.
@EXPORT_OK does export of symbols on demand basis.
"""
If this patch caused a conflict with a patch you wrote prior to its
push:
* Make sure you are not reintroducing a "use" statement that has been
removed
* "$subroutine" is not exported by the C4::$MODULE module
means that you need to add the subroutine to the @EXPORT_OK list
* Bareword "$subroutine" not allowed while "strict subs"
means that you didn't imported the subroutine from the module:
- use $MODULE qw( $subroutine list );
You can also use the fully qualified namespace: C4::$MODULE::$subroutine
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Also adds syspref to optionally choose to embed item information in record when adding/editing bibliographic and item records.
Test plan:
1. Enable OAI-PMH and OAI-PMH:AutoUpdateSets sysprefs
2. Create a biblio record with 1 item with CCODE of NFIC
3. In Administration > OAI Sets configuration create a set and define
the mapping:
Field: 952, Subfield: 8, Operator: is equal to, Value: FIC
4. Run the build_oai_sets.pl script:
cd misc/migration_tools
sudo koha-shell <instancename>
./build_oai_sets.pl -i
5. Check what items are in the OAI set:
sudo koha-mysql <instancename>
select * from oai_sets_biblios;
6. Observe your bib from step 2 is not in the set
7. [ITEM EDIT] Edit the item to have CCODE of FIC repeat step 5 and observe biblio is still not in OAI set
8. [ITEM ADD] Add a new item with CCODE of FIC and repeat step 5 and observe biblio
still not in OAI set
9. [ITEM DELETE] Delete one of your 2 items and repeat step 5 and observe biblio still
not in OAI set
10. Apply patch, update database and restart plack (instruction below on
the latter 2 actions):
cd installer/data/mysql
sudo koha-shell <instancename>
./updatedatabase.pl
sudo koha-plack --restart <instancename>
11. Enable new syspref OAI-PMH:AutoUpdateSetsEmbedItemData
12. [ITEM ADD] Repeat step 8 and observe biblio is now in the OAI set
13. [ITEM EDIT] Edit both items to have a CCODE of NFIC and repeat step 5 and
observe bib now not in set
14. [ITEM EDIT] Edit one of the items to have CCODE of FIC and repeat step 5
and observe bib now in set
15. [ITEM DELETE] Delete all items off bib and repeat step 5 and observe bib
no longer in set
Sponsored-By: Catalyst IT
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
We certainly faced 3 similar bugs due to this syntax: bug 23006, bug
22941 and bug 17526.
To prevent other issues related to this syntax this patch suggests to
replace them all in one go.
Test plan:
Confirm that the 2 syntaxes are similar
Eyeball the patch and confirm that there is no typo!
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Currently, an OAI-PMH set mapping will only match if the value it
is looking for occurs in the first instance of a repeated field.
To test:
- Apply the first patch with two new tests
- Run something like this:
sudo koha-shell -c "prove -v t/db_dependent/OAI/Sets.t" kohadev
- Verify that the last test fails
- Apply this secind patch
- Rerun the test file above
- Verify that all tests now pass
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Currently, the rules used to create OAI sets are processed with the 'or'
boolean operator between each rule.
This patch allows to use 'or' or 'and' between the rules.
The evaluation of the rules is done according to the boolean operators
precedence: AND has a higher precedence than OR.
For example:
A and B or C and D
will be evaluated as follow:
(A and B) or (C and D)
Test plan:
- Apply the patch
- Apply the atomicupdate
- Prove t/db_dependant/OAI/AndSets.t
- Check that existing mappings still work
- Try modifying existing mappings
- Try creating new mappings
- Check that the boolean operators precedence is correctly taken into account
Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch replaces the TRUNCATE statement in ModOAISetsBiblios by a
DELETE statement. A truncate will cause an implicit commit and will
therefore commit the transaction started in the test script.
Also simplifying the module load in the test script.
Test plan:
Do not apply this patch and observe that biblio records are added to your
database by running t/db_dependent/OAI/Sets.t.
Apply this patch, run the test again and verify that it does no longer
add records to your biblio table.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
http://bugs.koha-community.org/show_bug.cgi?id=9987
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Testing C4::OAI::Sets in t/db_dependent/OAI/Sets.t
Adding &DelOAISetsBiblio in @EXPORT at the beginning
of OAI/Sets.pm, it was the only one missing
TEST PLAN
---------
1. Apply patch
2. prove -v t/db_dependent/OAI/Sets.t
-- All 145 tests should run successfully without any error or warning
3. Run koha qa test tools
-- No issues found.
Additionally, I comfirmed why DelOAISetsBiblio was added in
the list of exported functions.
$ grep "^sub " C4/OAI/Sets.pm | sort > check1
$ for i in `grep "\&[A-Z]" C4/OAI/Sets.pm | xargs `; do echo $i; done | sort > check2
Of course the outputs have a little extra around them, so
$ vi check1
$ vi check2
$ diff check1 check2
Lo and behold, it was that one function. Necessary for 100% coverage.
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
In OAI set mappings, the value "is equal to" is hardcoded. This
enhancement changes it to a dropdown menu to choose between "is equal
to" and "not equal to".
To test:
* define a set
* define a mapping for said set with "is equal to"
* run /misc/migration_tools/build_oai_sets.pl -r -v
* confirm that you have correct entries in SQL: select * from
oai_sets_biblios;
* change mapping to 'not equal to', save
* run /misc/migration_tools/build_oai_sets.pl -r -v
* confirm that you have correct entries in SQL: select * from
oai_sets_biblios;
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Amended patch: Fix bug id in updatedb.pl
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
New sql tables:
- oai_sets: contains the list of sets, described by a spec and a name
- oai_sets_descriptions: contains a list of descriptions for each set
- oai_sets_mappings: conditions on marc fields to match for biblio to be
in a set
- oai_sets_biblios: list of biblionumbers for each set
New admin page: allow to configure sets:
- Creation, deletion, modification of spec, name and descriptions
- Define mappings which will be used for building oai sets
Implements OAI Sets in opac/oai.pl:
- ListSets, ListIdentifiers, ListRecords, GetRecord
New script misc/migration_tools/build_oai_sets.pl:
- Retrieve marcxml from all biblios and test if they belong to defined
sets. The oai_sets_biblios table is then updated accordingly
New system preference OAI-PMH:AutoUpdateSets. If on, update sets
automatically when a biblio is created or updated.
Use OPACBaseURL in oai_dc xslt
Following suggestion by Vincent Danjean for Debian
packaging, 0755 -> 0644 for non-executable
files.
Also removed shebang from a few modules in C4.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
