On bug 17591 we discovered that there was something weird going on with
the way we export and use subroutines/modules.
This patch tries to standardize our EXPORT to use EXPORT_OK only.
That way we will need to explicitely define the subroutine we want to
use from a module.
This patch is a squashed version of:
Bug 17600: After export.pl
Bug 17600: After perlimport
Bug 17600: Manual changes
Bug 17600: Other manual changes after second perlimports run
Bug 17600: Fix tests
And a lot of other manual changes.
export.pl is a dirty script that can be found on bug 17600.
"perlimport" is:
git clone https://github.com/oalders/App-perlimports.git
cd App-perlimports/
cpanm --installdeps .
export PERL5LIB="$PERL5LIB:/kohadevbox/koha/App-perlimports/lib"
find . \( -name "*.pl" -o -name "*.pm" \) -exec perl App-perlimports/script/perlimports --inplace-edit --no-preserve-unused --filename {} \;
The ideas of this patch are to:
* use EXPORT_OK instead of EXPORT
* perltidy the EXPORT_OK list
* remove '&' before the subroutine names
* remove some uneeded use statements
* explicitely import the subroutines we need within the controllers or
modules
Note that the private subroutines (starting with _) should not be
exported (and not used from outside of the module except from tests).
EXPORT vs EXPORT_OK (from
https://www.thegeekstuff.com/2010/06/perl-exporter-examples/)
"""
Export allows to export the functions and variables of modules to user’s namespace using the standard import method. This way, we don’t need to create the objects for the modules to access it’s members.
@EXPORT and @EXPORT_OK are the two main variables used during export operation.
@EXPORT contains list of symbols (subroutines and variables) of the module to be exported into the caller namespace.
@EXPORT_OK does export of symbols on demand basis.
"""
If this patch caused a conflict with a patch you wrote prior to its
push:
* Make sure you are not reintroducing a "use" statement that has been
removed
* "$subroutine" is not exported by the C4::$MODULE module
means that you need to add the subroutine to the @EXPORT_OK list
* Bareword "$subroutine" not allowed while "strict subs"
means that you didn't imported the subroutine from the module:
- use $MODULE qw( $subroutine list );
You can also use the fully qualified namespace: C4::$MODULE::$subroutine
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch renames the systempreference to be a bit clearer and uses
explicit options rather than a yes/no
Additionally it standardizes the export from the advanced cataloging editor
with that from the details page
To test:
1 - Apply patches
2 - Update database and restart all the things
3 - Open a record in the advanced editor and save it as marc and xml
4 - Note the file name is 'bib-{biblionumber.{format}'
5 - Edit the syspref 'DefaultSaveRecordFileID' to be control number
6 - Repeate 3-4 on a record with and without a control number
7 - If control number present fiule name should be 'record-{controlnumber}.{format}'
8 - Otherwise it should be as above
9 - Repeat tests from the details page of a record
10 - Repeat tests from the opac details page of a record
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
When exporting a record with diacritics in RIS
format from the detail page in OPAC the encoding
is not correct and the diacritics appear broken.
This patch fixes it.
To test:
- Find or create a record with some special
chars in your OPAC
- Go to the detail page
- Save as > RIS
- Verify the diacritics are broken in an editor
- Apply patch
- Repeat and verify everything now displays correctly
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
- Added missing GetHiddenItems parameter change case
Without this prove t had a failure.
- Always use mocks, not set_preference
- Tweaks so t/db_dependent/00-strict.t passes
There was a typo botcat vs borcat and borrowernumber was never
defined. Grabbing from userenv, like other code does.
- Tweak t/db_dependent/Items.t to fully test changes
This will test all the if structures fully in GetHiddenItemnumbers.
prove t/db_dependent/Items.t
- Tweak borrower category code
$borrower->{categorycode} on a Koha::Patron is not the
same as $borrower->categorycode. Fixed error.
- Search was returning URLS for wrong interface
There was one search context place wrong. Changed it to $is_opac
as the logic for setting $is_opac was modified correctly.
- Corrected issues with category code.
When a user isn't logged in, $borrower is undef and causes error
when determining category code. Added conditional check.
- Properly trigger all changes in C4/Search.pm
- Fix QA Test tool failures
C4/Search.pm had some tabs.
- Add some commenting to make sense of logic
- Refactor EmbedItemsInMarcBiblio parameters to hashref
- Trigger GetMarcBiblio's EmbedItemsInMarcBiblio call.
prove t/db_dependent/Items.t
- Add missing test to trigger Koha/BiblioUtils/Iterator change
- Add borrower category overrides
These files generally add borcat parameter to GetMarcBiblio.
Others might include correction of filtering of items
(opac-basket), or a comment as to why no changes were done
(opac-search).
In the case of opac-search, correcting the first FIXME will
likely correct the OpacHiddenItems issues on tags. As such,
that is beyond this bugs scope.
Some code had loop optimizations and fixes made, like a
'next unless $record' when the biblio shouldn't even be in
the list.
- Modify opac-ISBDdetail and opac-MARCdetail
Both files had similar logic. They were rearranged and
optimized, so that both files would have practically identical
initial blocks of code.
Optimizations were possible, because GetMarcBiblio
returns a filtered record, so that there is no double call
(once in the opac-### file and once in GetMarcBiblio) to
GetHiddenItemnumbers.
- Fix hiding in opac-tags
opac/opac-tags.pl was not properly hiding.
There is currently one known bug associated with tags left.
If you have two biblios tagged by different people with the
same tag, the opac-search will show the one you tagged that
is supposed to be hidden, because tag searches work differently
than regular searches. This is beyond the scope of this bug.
See the FIXME's in opac/opac-search.pl
- Trigger the C4::ILSDI::Services changes
prove t/db_dependent/ILSDI_Services.t
- Added missing 'my'
- Test C4/Labels/Label.pm changes
- Improve C4::Record::marcrecord2csv test cases
- Corrected opac-details searchResult call
- Fix breaking issues constraint in ITerator test
- Fix ILSDI_Services test when clubs with branch exist
- Rebased again!
- Rebased t/db_dependent/Items.t conflict.
The test plan is in comment #112 last I checked.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Change parameters to a hashref.
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Looks good to me.
Two calls in migration_tools/22_to_30 still in old style.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch adds the frameworkcode option param, using each record's frameworkcode
as expected by the filter. Otherwise the ViewPolicy filter falls back to the
default framework.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Koha::RecordProcessor and the defined filters are supposed to bring us
joy and happiness. Let's keep the code compact, simple and clean.
This patch removes record cloning all over the place.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
There are still some leaks, but it is not as a result
of the filter, but rather a result of poorly written
template files.
Bug fixing template files is beyond the scope of this
set of patches.
TEST PLAN
---------
1) Backup your DB
2) run the following SQL on your DB.
> UPDATE marc_subfield_structure set hidden=-8;
-- this should set EVERYTHING to hidden across the board.
3) In staff client, set OPACXSLTDetailsDisplay to blank
4) In OPAC, view any detail.
-- Normal view may mostly leak values still.
-- MARC view may leak values.
-- ISBD view may leak values.
5) In staff client, set OPACXSLTDetailsDisplay to default
6) In OPAC, view any detail.
-- same issues as step 4
-- 'View Plain' may leak too.
7) 'Save record' -> 'Dublin Core'
8) Apply this patch
9) run koha qa test tools
-- should be fine
10) prove -v t/db_dependent/Filter_MARC_ViewPolicy.t
-- should pass
-- this proves Koha/Filter/MARC/ViewPolicy.pm tweaks too
11) In OPAC, view any detail.
-- Normal view:
-- Material type comes from the LEADER field.
-- Lists this is on will still display
-- 'Tags from this library' will still display
-- Item information in table will still display
(THIS IS BEYOND SCOPE)
-- MARC view:
-- Record number is leaked
(THIS IS BEYOND SCOPE)
-- 'View plain' leaks LEADER field.
-- ISBD view may leak field headings, but not values.
(THIS IS BEYOND SCOPE)
12) In staff client, set OPACXSLTDetailsDisplay to blank
13) In OPAC, view any detail.
-- same kind of output as step 10
14) 'Save record' -> BIBTEXT
-- Should be next to nothing leaked.
15) 'Save record' -> Dublin Core
-- Should be the same or less leaked between the two versions.
-- (XML FILTERING IS BEYOND SCOPE)
16) In the staff client, go view the same record.
-- it should be mostly hidden in ISBD View.
17) run the following SQL on your DB.
> UPDATE marc_subfield_structure set hidden=1;
-- this should set EVERYTHING to hidden in OPAC, but not
the STAFF across the board.
18) Refresh the staff ISBD page
-- values should reappear.
19) View the ISBD details in the OPAC
-- values should still be hidden.
20) Check out the OPAC Cart and List
-- while the intermediate pages may still leak
the download links should leak very minimally.
-- (CARTS AND LISTS ARE BEYOND SCOPE, THOUGH
THE INTRANET ISBD AND SOME CART/LIST STUFF
WERE FIXED BECAUSE OF THE GetISBDView REFACTOR)
Expectations:
Before Patch - all the OPAC Detail pages will display things
After Patch - all the OPAC Detail pages will display much less,
and hopefully nothing (though there are known limits).
the ISBD detail page in the Staff client will be
filtered as well based on STAFF settings.
The saving/exporting should generate nearly empty
files.
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Applying the filtering and then...
Debugging opac/opac-detail.pl filtering
Debugging opac/opac-ISBDdetail.pl more
Debugging opac/opac-export.pl
Tweak opac/opac-export.pl fix variable declarations, conditional assignments
Debugging opac/opac-showmarc.pl
https://bugs.koha-community.org/show_bug.cgi?id=11592
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This simple patch fixes wide character warning raised by opac-export.pl when
'MARC (Unicode/UTF-8)' and 'MARC (Unicode/UTF-8, Standard)' formats are chosen for downloading records.
To test:
- Have records indexed
- Open your OPAC logs:
$ tail -f /var/log/koha/kohadev/opac-error.log
- Open the detail page in the opac for a record
- Choose Save record > MARC (Unicode/UTF-8)
=> FAIL: opac-export.pl: Wide character in print at /home/vagrant/kohaclone/opac/opac-export.pl line 116., referer: http://localh...
- Choose Save record > MARC (Unicode/UTF-8, Standard)
=> FAILE opac-export.pl: Wide character in print...
- Apply the patch
- Choose Save record > MARC (Unicode/UTF-8)
=> SUCCESS: No warnings raised.
- Choose Save record > MARC (Unicode/UTF-8, Standard)
=> SUCCESS: No warnings raised.
- Sign off :-D
Followed test plan, works as expected.
Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
When Koha export a bibliographic record to DC, makes it in XML format.
This XML not follows the DC-XML recommendations as should be: elements
in uppercase eg. dc:Date, dc:Creator, section 4.2, recommendation 4
explain that "The property names for the 15 DC elements should be all
lower-case." eg. dc:date, dc:creator" and section "4.3 Example - a
simple DC record", xsi:schemaLocation="http://example.org/myapp/http://example.org/myapp/schema.xsd" schema does not exist.
NOTE: This new feature implement the XSLT transformation for OAI-DC,
SRW-DC and RDF-DC
Test plan
---------------
1) Download Dublin Core file from record detail page. Open up the file,
and make sure that the document not follows the DC-XML
recommendations as should be.
2) Apply patch.
3) Go to whichever bib record in OPAC or staff and click on Save >
Dublin Core. A modal will display, prove all options.
4) Change the system preference 'Opac ExportOptions' by enabling and
disabling Dublin Core and try to download a record.
5) Try several bibliographic records in any format (book, magazine, DVD,
etc.) to confirm that properly exported.
6) Test with all marc flavours.
Sponsored-by: Universidad de El Salvador
Signed-off-by: Frederic Demians <f.demians@tamil.fr>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
http://bugs.koha-community.org/show_bug.cgi?id=9987
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This patch adds the option to export a record (from within the
OPAC record detail page) in the ISBD format.
To test:
- Enable ISBD on the OpacExportOptions syspref
- Look for a record on your opac
- On the detail page, notice there's a new ISBD option for 'Save record'
- Choose ISBD
=> SUCCESS: a file containing the ISBD format for the record is downloaded
Sponsored-by: Orex Digital
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
http://bugs.koha-community.org/show_bug.cgi?id=12174
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This patch solves the bug for UTF-8 output. MARC-8 works only if you save it as a file. In fact the standard Internet stack (browser, web server, etc.) doesn't know anything about MARC-8 or it's ISO base (ISO 2022).
To test the bug:
-- without the patch use Zotero on a recent Koha install with Apache.
You can use PTFS sandboxes. You can't use Biblibre sandboxes, they use Ngix.
-- Test a record with special chars.
-- Install the patch
-- Catch (using Zotero) the same record.
-- Test all directly download from Opac.
-- To check MARC-8, use a tool that support it. For example MarcEdit.
http://bugs.koha-community.org/show_bug.cgi?id=12174
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Work as described
Tested with zotero add-on for firefox.
Before patch bad diacritics, after patch ok
Record exported as MARC-8, transformed using
yaz-iconv -f marc8 -t utf-8 bib.marc8 > bib.utf8
yaz-marcdump -i marc -o line bib.utf8, checking accents,
also comparing with direct utf-8 export
No koha-qa errors
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Tested with Zotero in Firefox and with yaz-iconv as suggested by
Bernardo, no problems found.
Passes tests and QA script.
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
The opac-export.pl script had a number of XSS vulnerabilities relating
to its error handling.
To test:
1) Go to /cgi-bin/koha/opac-export.pl?op=export&bib=2&format=<h2>evil</h2>
(substituting a valid biblionumber for the '2')
2) Notice that "evil" is rendered as an h2 heading.
3) Apply patch.
4) Notice that you now see the h2 tags, and they are not rendered by
the browser.
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Add an option for marcstd to the opac-export.pl and catalogue/export.pl
scripts. This new format removes all 9XX, X9X, XX9 fields and subfield $9
(with the exception of 490 in flavours of MARC other than UNIMARC). The work is
done in C4::Record::marc2marc.
This patch adds the new export option 'marcstd' for exporting MARC
records without 9xx, x9x and xx9 fields and subfields to the staff
detail page.
Testing plan:
1. Export a record in "MARC (Unicode/UTF-8)" format as a control
2. In the OPAC, run the following jQuery to add the marcstd option to the UI:
> $("#export #format").append("<option value='marcstd'>MARC (no 9xx)</option>");
3. Export the same record in "MARC (no 9xx)" format
4. Compare the two, noticing that any subfield $9 or fields including 9 (other
than 490 in flavours of MARC other than UNIMARC) have been removed
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Works as advertised now.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
This patch squashes both the original patch and Katrin's follow-up adding
marcstd as an export option on the staff client.
Feb 13, 2012 (marcel): Amended this patch to resolved two definitions of $error in catalogue/export script.
Prior to this patch, RIS export was producing
effectively empty output.
Signed-off-by: Steven Callender <stevecallender@esilibrary.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Added check on returned marc. If record does not exist, generate 404. (Prevents 500 errors by Googlebot on deleted records.)
Fixed typo, whitespace. Removed if on op=export. Added check on format.
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Ian Walls <ian.walls@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
- The following export pages used to embed items when exporting,
this was no longer the case, so they were fixed :
Intranet :
- basket/downloadcart.pl,
- virtualshelves/downloadshelf.pl
- catalogue/export.pl
Opac :
- opac/opac-downloadcart.pl
- opac/opac-downloadshelf.pl
- opac/opac-export.pl
- Notes :
- GetMarcBiblio used to embed items data, this was no longer the case,
so an optional parameter was added to choose if items should be embedded or not.
This way, previous work on this bug is not broken, and this is a pretty usefull
feature, imho.
- An optional parameter has been added to SetUTF8Flag, to be able to use NFD during
normalization. This was required to make Unicode/UTF-8 export work again.
Signed-off-by: Claire Hernandez <claire.hernandez@biblibre.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Most Perl scripts (as opposed to modules) do
not need to require Exporter.
No user-visible or documentation changes.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
HTML::Template is no more used, some were remaining,
fixing the "use ...;" to H::T::Pro only
Signed-off-by: Chris Cormack <crc@liblime.com>
Signed-off-by: Joshua Ferraro <jmf@liblime.com>
