Browse Source

Bug 5478 - Automate MARC framework import/export

Module to Import/Export a Framework structure to CSV/Excel-xml/ODS/SQL in Intranet Administration - MARC Frameworks section.
    There are two new links: "Export" to export to a format; and "Import" to import from a file.
    The data exported/imported is the one stored in the MySQL tables marc_tag_structure, marc_subfield_structure.

    Exported works as follows:
    1) CSV: As this format only allows one worksheet, the data from the tables is splitted with a row with #-# cells or with the
    names of the fields of the next MySQL table. Each row has as much cells as fields has the MySQL table. The first row contains the
    field names, the remaining holds the data.
    2) Excel: Excel xml 2003 format. Each MySQL table has its own worksheet in the spreadsheet. Rows and cells data as CSV.
    3) ODS: OpenDocument Spreadsheet compressed format, creates a temporary directory to generate the files needed to create the zip file.
    Each MySQL table has its own worksheet in the spreadsheet. Rows and cells data as CSV.
    4) SQL: Text file, the first row for each table is a delete and the remaining are inserts.

    Importing reads the rows from the spreadsheet/text-file as follows:
    1) CSV: Each row inserts or updates the associated MySQL table for this framework. At the end of the importing for a MySQL table, deletes the rows in the database that don't possess a correspondence with the spreadsheet.
    2) Excel: Imports each worksheet to the associated MySQL table. Works as the CSV for each worksheet.
    3) ODS: Creates a temporary directory to decompress and read the content.xml. This file has the data needed to import.
    Works as the CSV for each worksheet.
    4) Executes the SQL file.
    If the file imported has a different frameworkcode that the framework importing, the framecode is changed along the process.

    The Csv format will be the default.
    It uses perl module Archive::Zip or zip/unzip system command to process ODS files.
    To parse the sql files when importing it uses SQL::Statement or homemade parsing.

Signed-off-by: Nicole C. Engard <nengard@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
3.4.x
Salvador Zaragoza Rubio 13 years ago
committed by Chris Cormack
parent
commit
689bc19154
  1. 1358
      C4/ImportExportFramework.pm
  2. 77
      admin/import_export_framework.pl
  3. 56
      koha-tmpl/intranet-tmpl/prog/en/css/staff-global.css
  4. 136
      koha-tmpl/intranet-tmpl/prog/en/modules/admin/biblio_framework.tmpl

1358
C4/ImportExportFramework.pm

File diff suppressed because it is too large

77
admin/import_export_framework.pl

@ -0,0 +1,77 @@
#!/usr/bin/perl
# Copyright 2010-2011 MASmedios.com y Ministerio de Cultura
#
# This file is part of Koha.
#
# Koha is free software; you can redistribute it and/or modify it under the
# terms of the GNU General Public License as published by the Free Software
# Foundation; either version 2 of the License, or (at your option) any later
# version.
#
# Koha is distributed in the hope that it will be useful, but WITHOUT ANY
# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
# A PARTICULAR PURPOSE. See the GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
# with Koha; if not, write to the Free Software Foundation, Inc.,
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
use strict;
use warnings;
use CGI;
use C4::Context;
use C4::ImportExportFramework;
my $input = new CGI;
my $frameworkcode = $input->param('frameworkcode') || '';
my $action = $input->param('action') || 'export';
## Exporting
if ($action eq 'export' && $input->request_method() eq 'GET') {
my $strXml = '';
my $format = $input->param('type_export_' . $frameworkcode);
ExportFramework($frameworkcode, \$strXml, $format);
if ($format eq 'csv') {
# CSV file
print $input->header(-type => 'application/vnd.ms-excel', -attachment => 'export_' . $frameworkcode . '.csv');
print $strXml;
} elsif ($format eq 'sql') {
# SQL file
print $input->header(-type => 'text/plain', -attachment => 'export_' . $frameworkcode . '.sql');
print $strXml;
} elsif ($format eq 'excel') {
# Excel-xml file
print $input->header(-type => 'application/excel', -attachment => 'export_' . $frameworkcode . '.xml');
print $strXml;
} else {
# ODS file
my $strODS = '';
createODS($strXml, 'en', \$strODS);
print $input->header(-type => 'application/vnd.oasis.opendocument.spreadsheet', -attachment => 'export_' . $frameworkcode . '.ods');
print $strODS;
}
## Importing
} elsif ($input->request_method() eq 'POST') {
my $ok = -1;
my $fieldname = 'file_import_' . $frameworkcode;
my $filename = $input->param($fieldname);
# upload the input file
if ($filename && $filename =~ /\.(csv|ods|xml|sql)$/i) {
my $extension = $1;
my $uploadFd = $input->upload($fieldname);
if ($uploadFd && !$input->cgi_error) {
my $tmpfilename = $input->tmpFileName($input->param($fieldname));
$filename = $tmpfilename . '.' . $extension; # rename the tmp file with the extension
$ok = ImportFramework($filename, $frameworkcode, 1) if (rename($tmpfilename, $filename));
}
}
if ($ok >= 0) { # If everything went ok go to the framework marc structure
print $input->redirect( -location => '/cgi-bin/koha/admin/marctagstructure.pl?frameworkcode=' . $frameworkcode);
} else {
# If something failed go to the list of frameworks and show message
print $input->redirect( -location => '/cgi-bin/koha/admin/biblio_framework.pl?error_import_export=' . $frameworkcode);
}
}

56
koha-tmpl/intranet-tmpl/prog/en/css/staff-global.css

@ -1933,3 +1933,59 @@ ul.budget_hierarchy li:first-child:after {
margin : 1em 0;
}
fieldset.rows+h3 {clear:both;padding-top:.5em;}
.import_export{
position:relative;
}
.import_export .import_export_options{
background: white;
border: 1px solid #CDCDCD;
left: 60px;
padding: 10px;
position: absolute;
top: 0;
z-index: 1;
width: 300px;
}
.import_export_options li{
display: block;
list-style: none;
padding-top: 10px;
}
.import_export_options .import_export_close {
cursor: pointer;
text-decoration: underline;
}
.import_export_options .export_ok {
padding: 10;
background: #E3E3E3 none;
cursor: pointer;
margin-left: 20px;
border: none;
}
.import_export_options .import_ok {
padding: 10;
background: #E3E3E3 none;
cursor: pointer;
margin-left: 20px;
border: none;
}
.form_import .input_import {
border: 1px solid #bcbcbc;
}
.import_export_options .importing {
padding: inherit;
background: none;
}
.li_close_import_export {
text-align: right;
}
.importing {
position: relative;
}
.importing .importing_msg {
padding-left: 10px;
padding-bottom: 10px;
}

136
koha-tmpl/intranet-tmpl/prog/en/modules/admin/biblio_framework.tmpl

@ -48,6 +48,79 @@ $(document).ready(function() {
});
//]]>
</script>
<script type="text/javascript">
/* Import/Export from/to spreadsheet */
var importing = false;
$(document).ready(function() {
$("body").css("cursor", "auto");
$('.import_export_options').hide();
$('a.import_export_fw').click(function() {
if (!importing) {
$('.import_export_options').hide();
$(this).next().show('slide');
}
return false;
});
$('.import_export_close').click(function() {
if (!importing) {
$('.import_export_options').fadeOut('fast');
$("body").css("cursor", "auto");
}
});
$('.input_import').val("");
var matches = new RegExp("\\?error_import_export=(.+)$").exec(window.location.search);
if (matches && matches.length > 1) {
alert("Error importing the framework " + decodeURIComponent(matches[1]));
}
});
$(function() {
$('input.input_import').change( function() {
var filename = $(this).val();
if ( ! /(?:\.csv|\.sql|\.ods|\.xml)$/.test(filename)) {
$(this).css("background-color","yellow");
alert('Please select an ods or xml file');
$(this).val("");
$(this).css("background-color","white");
}
});
$('form.form_export').submit(function() {
$('.import_export_options').hide();
return true;
});
$('form.form_import').submit(function() {
var id = $(this).attr('id');
var obj = $('#' + id + ' input:file');
if (/(?:\.csv|\.sql|\.ods|\.xml)$/.test(obj.val())) {
if (confirm('Do you really want to import the framework fields/subfields (will overwrite current configuration, for safety reasons please make before an export to have a backup file)?')) {
var frameworkcode = $('#' + id + ' input:hidden[name=frameworkcode]').val();
$('#importing_' + frameworkcode).find("span").html("Importing <strong>" + frameworkcode + "</strong> from <i>" + obj.val().replace(new RegExp("^.+[/\\\\]"),"") + "</i>");
if (navigator.userAgent.toLowerCase().indexOf('msie') != -1) {
var timestamp = new Date().getTime();
$('#importing_' + frameworkcode).find("img").attr('src', '/intranet-tmpl/prog/img/loading.gif' + '?' +timestamp);
}
$('#importing_' + frameworkcode).css('display', 'block');
if (navigator.userAgent.toLowerCase().indexOf('firefox') == -1) $("body").css("cursor", "progress");
importing = true;
return true;
} else
return false;
}
obj.css("background-color","yellow");
alert('Please select an spreadsheet (csv, ods, xml) or sql file');
obj.val("");
obj.css("background-color","white");
return false;
});
});
</script>
</head>
<body>
<!-- TMPL_INCLUDE NAME="header.inc" -->
@ -115,6 +188,8 @@ $(document).ready(function() {
<th>&nbsp;</th>
<th>Edit</th>
<th>Delete</th>
<th title="Export framework structure (fields, subfields) to a spreadsheet file (.csv, .xml, .ods) or SQL file">Export</th>
<th title="Import framework structure (fields, subfields) from a spreadsheet file (.csv, .xml, .ods) or SQL file">Import</th>
</tr>
<tr>
<td>&nbsp;</td>
@ -122,6 +197,36 @@ $(document).ready(function() {
<td><a href="marctagstructure.pl?frameworkcode=<!-- TMPL_VAR NAME="frameworkcode" -->">MARC structure</a></td>
<td>&nbsp;</td>
<td>&nbsp;</td>
<td><div class="import_export"><a class="import_export_fw" href="#" title="Export <!-- TMPL_VAR NAME="frameworkcode" --> framework structure (fields, subfields) to a spreadsheet file (.csv, .xml, .ods) or SQL file">Export</a>
<div class="import_export_options">
<form action="import_export_framework.pl" name="form_<!-- TMPL_VAR NAME="frameworkcode" -->" method="get" target="_blank" class="form_export">
<input type="hidden" name="frameworkcode" value="<!-- TMPL_VAR NAME="frameworkcode" -->" />
<ul>
<li class="li_close_import_export"><span class="import_export_close" title="Close popup">Close</span></li>
<li><input type="radio" name="type_export_<!-- TMPL_VAR NAME="frameworkcode" -->" value="csv" id="type_export_<!-- TMPL_VAR NAME="frameworkcode" -->" checked="checked" /><label for="type_export_<!-- TMPL_VAR NAME="frameworkcode" -->" title="Export to CSV Spreadsheet">Export to CSV Spreadsheet</label></li>
<li><input type="radio" name="type_export_<!-- TMPL_VAR NAME="frameworkcode" -->" value="excel" id="type_export_<!-- TMPL_VAR NAME="frameworkcode" -->" /><label for="type_export_<!-- TMPL_VAR NAME="frameworkcode" -->" title="Export to Excel as xml format, compatible with OpenOffice/LibreOffice as well">Export to Excel with xml format</label></li>
<li><input type="radio" name="type_export_<!-- TMPL_VAR NAME="frameworkcode" -->" value="ods" id="type_export_<!-- TMPL_VAR NAME="frameworkcode" -->" /><label for="type_export_<!-- TMPL_VAR NAME="frameworkcode" -->">Export to OpenDocument Spreadsheet format</label></li>
<li><input type="radio" name="type_export_<!-- TMPL_VAR NAME="frameworkcode" -->" value="sql" id="type_export_<!-- TMPL_VAR NAME="frameworkcode" -->" /><label for="type_export_<!-- TMPL_VAR NAME="frameworkcode" -->" title="Export to SQL">Export to SQL</label></li>
<li><input type="submit" class="export_ok" href="#" value="Export" title="Export to a spreadsheet" /></li>
</ul>
</form>
</div>
</div>
</td>
<td><div class="import_export"><a href="#" class="import_export_fw" title="Import <!-- TMPL_VAR NAME="frameworkcode" --> framework structure (fields, subfields) from a spreadsheet file (.csv, .xml, .ods) or SQL file">Import</a>
<div class="import_export_options">
<form action="/cgi-bin/koha/admin/import_export_framework.pl" name="form_i_<!-- TMPL_VAR NAME="frameworkcode" -->" id="form_i_<!-- TMPL_VAR NAME="frameworkcode" -->" method="post" enctype="multipart/form-data" class="form_import">
<input type="hidden" name="frameworkcode" value="<!-- TMPL_VAR NAME="frameworkcode" -->" />
<input type="hidden" name="action" value="import" />
<ul>
<li class="li_close_import_export"><span class="import_export_close" title="Close popup">Close</span></li>
<li><label for="file_import_<!-- TMPL_VAR NAME="frameworkcode" -->" title="Import from a spreadsheet, formats available: ods, xml (formatted from excel)">Spreadsheet file</label><input type="file" name="file_import_<!-- TMPL_VAR NAME="frameworkcode" -->" id="file_import_<!-- TMPL_VAR NAME="frameworkcode" -->" class="input_import" value="" autocomplete="off" /></li>
<li><input type="submit" class="import_ok" value="Import" title="Import from a spreadsheet" /><div id="importing_<!-- TMPL_VAR NAME="frameworkcode" -->" style="display:none" class="importing"><img src="/intranet-tmpl/prog/img/loading.gif" /><span class="importing_msg"></span></div></li>
</ul>
</form>
</div>
</div>
</td>
</tr>
<!-- note highlight assignment appears backwards because we already have a normal row for Default -->
<!-- TMPL_LOOP NAME="loop" -->
@ -133,6 +238,37 @@ $(document).ready(function() {
<td><a href="marctagstructure.pl?frameworkcode=<!-- TMPL_VAR name="frameworkcode" -->" >MARC structure</a></td>
<td><a href="<!-- TMPL_VAR name="script_name" -->?op=add_form&amp;frameworkcode=<!-- TMPL_VAR name="frameworkcode" escape="HTML" -->">Edit</a></td>
<td><a href="<!-- TMPL_VAR name="script_name" -->?op=delete_confirm&amp;frameworkcode=<!-- TMPL_VAR name="frameworkcode" escape="HTML" -->">Delete</a></td>
<td>
<div class="import_export"><a class="import_export_fw" href="#" title="Export <!-- TMPL_VAR NAME="frameworkcode" --> framework structure (fields, subfields) to a spreadsheet file (.csv, .xml, .ods) or SQL file">Export</a>
<div class="import_export_options">
<form action="import_export_framework.pl" name="form_<!-- TMPL_VAR NAME="frameworkcode" -->" method="get" target="_blank" class="form_export">
<input type="hidden" name="frameworkcode" value="<!-- TMPL_VAR NAME="frameworkcode" -->" />
<ul>
<li class="li_close_import_export"><span class="import_export_close" title="Close popup">Close</span></li>
<li><input type="radio" name="type_export_<!-- TMPL_VAR NAME="frameworkcode" -->" value="csv" id="type_export_<!-- TMPL_VAR NAME="frameworkcode" -->" checked="checked" /><label for="type_export_<!-- TMPL_VAR NAME="frameworkcode" -->" title="Export to CSV Spreadsheet">Export to CSV Spreadsheet</label></li>
<li><input type="radio" name="type_export_<!-- TMPL_VAR NAME="frameworkcode" -->" value="excel" id="type_export_<!-- TMPL_VAR NAME="frameworkcode" -->" /><label for="type_export_<!-- TMPL_VAR NAME="frameworkcode" -->" title="Export to Excel as xml format, compatible with OpenOffice/LibreOffice as well">Export to Excel as xml format</label></li>
<li><input type="radio" name="type_export_<!-- TMPL_VAR NAME="frameworkcode" -->" value="ods" id="type_export_<!-- TMPL_VAR NAME="frameworkcode" -->" /><label for="type_export_<!-- TMPL_VAR NAME="frameworkcode" -->">Export to OpenDocument Spreadsheet format</label></li>
<li><input type="radio" name="type_export_<!-- TMPL_VAR NAME="frameworkcode" -->" value="sql" id="type_export_<!-- TMPL_VAR NAME="frameworkcode" -->" /><label for="type_export_<!-- TMPL_VAR NAME="frameworkcode" -->" title="Export to SQL">Export to SQL</label></li>
<li><input type="submit" class="export_ok" href="#" value="Export" title="Export to a spreadsheet" /></li>
</ul>
</form>
</div>
</div>
</td>
<td><div class="import_export"><a class="import_export_fw" href="#" title="Import <!-- TMPL_VAR NAME="frameworkcode" --> framework structure (fields, subfields) from a spreadsheet file (.csv, .xml, .ods) or SQL file">Import</a>
<div class="import_export_options">
<form action="/cgi-bin/koha/admin/import_export_framework.pl" name="form_i_<!-- TMPL_VAR NAME="frameworkcode" -->" id="form_i_<!-- TMPL_VAR NAME="frameworkcode" -->" method="post" enctype="multipart/form-data" class="form_import">
<input type="hidden" name="frameworkcode" value="<!-- TMPL_VAR NAME="frameworkcode" -->" />
<input type="hidden" name="action" value="import" />
<ul>
<li class="li_close_import_export"><span class="import_export_close" title="Close popup">Close</span></li>
<li><label for="file_import_<!-- TMPL_VAR NAME="frameworkcode" -->" title="Import from a spreadsheet, formats available: ods, xml (formatted from excel)">Spreadsheet file</label><input type="file" name="file_import_<!-- TMPL_VAR NAME="frameworkcode" -->" id="file_import_<!-- TMPL_VAR NAME="frameworkcode" -->" class="input_import" value="" autocomplete="off" /></li>
<li><input type="submit" class="import_ok" value="Import" title="Import from a spreadsheet" /><div id="importing_<!-- TMPL_VAR NAME="frameworkcode" -->" style="display:none" class="importing"><img src="/intranet-tmpl/prog/img/loading.gif" /><span class="importing_msg"></span></div></li>
</ul>
</form>
</div>
</div>
</td>
</tr>
<!-- /TMPL_LOOP -->
</table>

Loading…
Cancel
Save