This patch fixes the comments from Comment 29 and the import
functionality. You should now be able to import an exported file without
editing the file at all and the authority type code will be overwritten
in the file (same behaviour as biblio frameworks).
Sponsored-by: Catalyst IT
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Mazen Khallaf <mazen.i.khallaf@gamil.com>
Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
This patch amends C4::ImportExportFramework to work for authority types as well as MARC frameworks.
New file: admin/import_export_authtype.pl
Update: Ensuring we are passing the right column to the right tables.
Update2: Making the error messages the same to be consistent with patch on Bug 15665
Update3: Fixing merge conflicts
Update4: Fixing merge conflicts and removing tabs
Update5: Getting rid of warns, making sure Import and Export of default
authority will work
Update6: Merge conflicts and making sure export of default auth type
works
Update7: Fixing merge conflicts and updating buttons to bootstrap3
To test:
1) Go to Admin -> Authority types
2) Confirm there are two new columns 'Export' and 'Import' in the table
3) Click 'Export' on an existing authority type and choose a file type, click 'Export'
4) Confirm that the authority type is exported as your chosen file type. Save the file
5) Create a new authority type
6) Import into your new authority type using the file you just exported
7) Confirm you are taken to auth_tag_structure.pl
8) Go back to Authority types
9) Export your new authority type. View the exported file and confirm
the authtypecode has been updated to match the code you set for the new
auth type
10) Click 'Import' next to any existing authority type and attempt to import a file that is not XML, CSV or ODS. Confirm that this fails and you are asked to import a file of the correct file type
11) Confirm Export and Import work for the Default authority type
12) Go to Admin -> MARC bibliographic framework
13) Confirm that the 'Export' and 'Import' functions still work here as well
Sponsored-by: Catalyst IT
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Mazen Khallaf <mazen.i.khallaf@gamil.com>
Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
The previous patch makes check_cookie_auth return the session instead of
$sessionID, so we are adjusting the different calls to prevent
confusion.
However they are mainly used to check the authentication status and
don't care about this second variable.
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
On bug 17591 we discovered that there was something weird going on with
the way we export and use subroutines/modules.
This patch tries to standardize our EXPORT to use EXPORT_OK only.
That way we will need to explicitely define the subroutine we want to
use from a module.
This patch is a squashed version of:
Bug 17600: After export.pl
Bug 17600: After perlimport
Bug 17600: Manual changes
Bug 17600: Other manual changes after second perlimports run
Bug 17600: Fix tests
And a lot of other manual changes.
export.pl is a dirty script that can be found on bug 17600.
"perlimport" is:
git clone https://github.com/oalders/App-perlimports.git
cd App-perlimports/
cpanm --installdeps .
export PERL5LIB="$PERL5LIB:/kohadevbox/koha/App-perlimports/lib"
find . \( -name "*.pl" -o -name "*.pm" \) -exec perl App-perlimports/script/perlimports --inplace-edit --no-preserve-unused --filename {} \;
The ideas of this patch are to:
* use EXPORT_OK instead of EXPORT
* perltidy the EXPORT_OK list
* remove '&' before the subroutine names
* remove some uneeded use statements
* explicitely import the subroutines we need within the controllers or
modules
Note that the private subroutines (starting with _) should not be
exported (and not used from outside of the module except from tests).
EXPORT vs EXPORT_OK (from
https://www.thegeekstuff.com/2010/06/perl-exporter-examples/)
"""
Export allows to export the functions and variables of modules to user’s namespace using the standard import method. This way, we don’t need to create the objects for the modules to access it’s members.
@EXPORT and @EXPORT_OK are the two main variables used during export operation.
@EXPORT contains list of symbols (subroutines and variables) of the module to be exported into the caller namespace.
@EXPORT_OK does export of symbols on demand basis.
"""
If this patch caused a conflict with a patch you wrote prior to its
push:
* Make sure you are not reintroducing a "use" statement that has been
removed
* "$subroutine" is not exported by the C4::$MODULE module
means that you need to add the subroutine to the @EXPORT_OK list
* Bareword "$subroutine" not allowed while "strict subs"
means that you didn't imported the subroutine from the module:
- use $MODULE qw( $subroutine list );
You can also use the fully qualified namespace: C4::$MODULE::$subroutine
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
It gives incomplete results and not sure it ever worked.
To prevent data loss it is better to remove this option.
Note that the code in C4/ImportExportFramework.pm is too complex to
attach with this patch to deletion of the code related to the XML
import/export. Safer to keep it for a separate bug report.
Test plan:
Import/export biblio frameworks and confirm that there is no more
reference to the xml format.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
So far the administration module only allowed for 2 permissions:
- circulation conditions (manage_circ_rules)
- everything else (parameters_remaining_permissions)
With this patch almost every section of the administration page
will have its own granular permission.
To test:
- Create different staff users:
1) One with parameters_remaining_permissions
2) One with parameters
3) One with catalogue and no parameters
4) One superlibrarian
- Apply the patch
- Run the database update
- Check the staff users:
1) All subpermissions, but manage_circ_rules
should be checked
2) Nothing should have changed
3) manage_item_serach_fields shoudl be checked
(page had catalogue permission before)
4) Nothing should have changed
- Try different settings of the permissions and
verify that
- Administration page behaves correctly
- Administration menu behaves correctly
! You shoudl only see what you have permission for
https://bugs.koha-community.org/show_bug.cgi?id=14391
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Test plan:
Check use warnings; use strict doesn't exist and use Modern::Perl is written instead in the following files:
admin/admin-home.pl
admin/aqcontract.pl
admin/aqplan.pl
admin/auth_tag_structure.pl
admin/branch_transfer_limits.pl
admin/check_budget_parent.pl
admin/check_parent_total.pl
admin/checkmarc.pl
admin/classsources.pl
admin/clone-rules.pl
admin/didyoumean.pl
admin/edi_accounts.pl
admin/edi_ean_accounts.pl
admin/import_export_framework.pl
admin/item_circulation_alerts.pl
admin/marctagstructure.pl
admin/matching-rules.pl
admin/printers.pl
admin/smart-rules.pl
admin/systempreferences.pl
admin/transport-cost-matrix.pl
Signed-off-by: Jon Knight <J.P.Knight@lboro.ac.uk>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Once again, after bug 16154 and bug 16259 we need to remove more
occurrence of CGi->param called in list context.
Refer to bug 15809 for more information.
Test plan:
Make sure you do not see the error on the modified scripts.
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
With the frameworkcode changed to an empty string, which really
is closer to the physical reality, some javascript somewhere
breaks. This means the spinner does not appear when you confirm
that you wish to import it.
TEST PLAN
----------
1) apply first patch
2) export default framework, doesn't matter format
3) import default framework using that file.
4) answer yes or whatever it is.
-- spinner does not appear.
5) apply this patch
6) import default framework use the same file.
7) answer yes or whatever it is.
-- spinner does appear.
8) decide if you wish to find and fix the javascript conditions.
if yes -- do so, and then obsolete this once you have.
if not -- sign this off after running koha qa test tools
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Caused by bug 16035.
Recreate the issue:
Go to Home › Administration › MARC frameworks
For the default framework: Actions > import
It will not work
Test plan:
Confirm test plans from bug 16035 and bug 17389 still pass
Confirm that the import for the default framework now works
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Trivial patch, no need to explain anything :)
Test plan:
Export default framework
Export other frameworks
=> The frameworkcode column should contain the framework code
Signed-off-by: Hector Castro <hector.hecaxmmx@gmail.com>
Works as advertised
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
This patch makes sure that, if there is no framework code,
$frameworkcode is set to 'default'. This fixes the format issue.
Then, if $frameworkcode = default, an empty string (rather than
null) is passed to the ExportFramework method, as the framework
code in the DB for default codes is "".
To test:
1) Go to Admin -> MARC Bibliographic framework
2) Try to export default framework in all formats, notice the file is
only exported as .ods and is an empty file. Notice warns in intranet
error log.
3) Apply patch and refresh page
4) Confirm exporting default framework works in all formats and
spreadsheet has data
5) Notice warns in error log are gone
6) Confirm export still works for other frameworks
Sponsored-by: Catalyst IT
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
http://bugs.koha-community.org/show_bug.cgi?id=9987
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Currently both the import_export_framework.pl script outputs data with
Perl's default encoding, ISO-8859. This patch properly sets
the binmode to UTF-8 when exporting SQL and CSV files using the PerlIO
layer (":encoding(UTF-8)") for STDOUT.
To test:
Export step test
- Use some ASCII character(s) with DIACRITICS in some field description
in a chosen framework.
- Export the framework at Administration > MARC frameworks
- Run this to check the file is ISO-8859 encoded:
$ file export_XXX.csv
export_XXX.csv: ISO-8859 text, with very long lines
(Note: try SQL and other output formats too. But not ODS)
- Apply the patch
- Export the framework again (change the name), and test encoding:
$ file export_XXX_2.csv
export_XXX_2.csv: UTF-8 Unicode text
Import step test
I assume you have two files, export_XXX.csv (ISO-8859 encoded) and
export_XXX_2.csv (XXX will depend on your framework's code)
- Reset your testing branch to master
- Import export_XXX.csv
- The string with non-ASCII chars is truncated at the first non-ASCII
char's position (Note: this is the current behaviour).
- Import export_XXX_2.csv
- The non-ASCII chars are broken, the logs show errors on non-UNICODE
chars. (Note: even thou UTF-8 is the expected encoding it is
treated as ISO-8859).
- Apply the patch
- Import the good (UTF-8 as expected) file and check everything worked
as expected.
No double encoding should occur with either combination of formats.
Sponsored-by: Universidad Nacional de Cordoba
Signed-off-by: Magnus Enger <digitalutvikling@gmail.com>
I put some Norwegian and accented letters in a fremawork to test.
Before the patch, the exported CSV came out as ISO-8859, after the
patch it came out as UTF-8. ODS and XML (viewed in LibreOffice)
both looked good, before and after the patch.
Importing the ISO-8859 CSV cut off the strings at the first non-ASCII
char. Importing the UTF-8 CSV worked as epected.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as expected, passes tests and QA script.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
The SQL option for MARC framework imports was subject to a bug whereby
somebody could use it to gain access to arbitrary information in the
database by uploading an SQL file containing unexpected statements.
As it is difficult to securely sanitize SQL, this patch removes the
option to use SQL as an import or export format.
To test:
[1] Verify that SQL no longer appears as an import or export option
for the MARC frameworks.
[2] Verify that exports and imports in CSV, Excel XML, and ODS formats
still work.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Works as advertised. The UI doesn't offer exporting/importing in the SQL format.
Crafting the URL to export SQL fallbacks to a spreadsheet format (ODS).
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as described, passes all tests and QA script.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch makes the MARC framework import/export script require
that the staff user be logged in with appropriate permissions for
managing the MARC frameworks.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
I can confirm the bug and the solution. After applying the patch
downloading the file without logging in first is no longer possible.
Also passes tests and QA script.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Module to Import/Export a Framework structure to CSV/Excel-xml/ODS/SQL in Intranet Administration - MARC Frameworks section.
There are two new links: "Export" to export to a format; and "Import" to import from a file.
The data exported/imported is the one stored in the MySQL tables marc_tag_structure, marc_subfield_structure.
Exported works as follows:
1) CSV: As this format only allows one worksheet, the data from the tables is splitted with a row with #-# cells or with the
names of the fields of the next MySQL table. Each row has as much cells as fields has the MySQL table. The first row contains the
field names, the remaining holds the data.
2) Excel: Excel xml 2003 format. Each MySQL table has its own worksheet in the spreadsheet. Rows and cells data as CSV.
3) ODS: OpenDocument Spreadsheet compressed format, creates a temporary directory to generate the files needed to create the zip file.
Each MySQL table has its own worksheet in the spreadsheet. Rows and cells data as CSV.
4) SQL: Text file, the first row for each table is a delete and the remaining are inserts.
Importing reads the rows from the spreadsheet/text-file as follows:
1) CSV: Each row inserts or updates the associated MySQL table for this framework. At the end of the importing for a MySQL table, deletes the rows in the database that don't possess a correspondence with the spreadsheet.
2) Excel: Imports each worksheet to the associated MySQL table. Works as the CSV for each worksheet.
3) ODS: Creates a temporary directory to decompress and read the content.xml. This file has the data needed to import.
Works as the CSV for each worksheet.
4) Executes the SQL file.
If the file imported has a different frameworkcode that the framework importing, the framecode is changed along the process.
The Csv format will be the default.
It uses perl module Archive::Zip or zip/unzip system command to process ODS files.
To parse the sql files when importing it uses SQL::Statement or homemade parsing.
Signed-off-by: Nicole C. Engard <nengard@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
