misc/translator/translate was doing three different things: - extract translatable strings - create or update PO files - install translated templates This patch separates responsibilities by moving the string extraction code into several 'xgettext-like' scripts and adds gulp tasks to automate string extraction and PO files update This has several benefits: - gulp runs tasks in parallel, so it's a lot faster (updating all PO files is at least 10 times faster with my 4-cores CPU) - there is no need for $KOHA_CONF to be defined LangInstaller.pm relied on $KOHA_CONF to get the different paths needed. I'm not sure why, since string extraction and PO update should work on source files, not installed files - string extraction code can be more easily tested This patch also brings a couple of fixes and improvements: - TT string extraction (strings wrapped in [% t(...) %]) was done with Template::Parser and PPI, which was extremely slow, and had some problems (see bug 24797). This is now done with Locale::XGettext::TT2 (new dependency) which is a lot faster, and fixes bug 24797 - Fix header in 4 PO files For backward compatibility, 'create' and 'update' commands of misc/translator/translate can still be used and will execute the corresponding gulp task Test plan: 1. Run `yarn install` and install Locale::XGettext::TT2 2. Run `gulp po:update` 3. Verify the contents of updated PO files 4. Run `cd misc/translator && ./translate install <lang>` 5. Verify that all (templates, sysprefs, xslt, installer files) is correctly translated 6. Run `gulp po:create --lang <lang>` and verify that it created all PO files for that language 7. Run `prove t/misc/translator` Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Need to install yarn & gulp, no errors Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>tags/v20.11.00
@@ -146,6 +146,7 @@ recommends 'Gravatar::URL', '1.03'; | |||
recommends 'HTTPD::Bench::ApacheBench', '0.73'; | |||
recommends 'LWP::Protocol::https', '5.836'; | |||
recommends 'Lingua::Ispell', '0.07'; | |||
recommends 'Locale::XGettext::TT2', '0.7'; | |||
recommends 'Module::Bundled::Files', '0.03'; | |||
recommends 'Module::Load::Conditional', '0.38'; | |||
recommends 'Module::Pluggable', '3.9'; | |||
@@ -157,7 +158,6 @@ recommends 'Net::SFTP::Foreign', '1.73'; | |||
recommends 'Net::Server', '0.97'; | |||
recommends 'Net::Z3950::SimpleServer', '1.15'; | |||
recommends 'PDF::FromHTML', '0.31'; | |||
recommends 'PPI', '1.215'; | |||
recommends 'Parallel::ForkManager', '0.75'; | |||
recommends 'Readonly', '0.01'; | |||
recommends 'Readonly::XS', '0.01'; | |||
@@ -0,0 +1,121 @@ | |||
# Internationalization | |||
This page documents how internationalization works in Koha. | |||
## Making strings translatable | |||
There are several ways of making a string translatable, depending on where it | |||
is located | |||
### In Template::Toolkit files (`*.tt`) | |||
The simplest way to make a string translatable in a template is to do nothing. | |||
Templates are parsed as HTML files and almost all text nodes are considered as | |||
translatable strings. This also includes some attributes like `title` and | |||
`placeholder`. | |||
This method has some downsides: you don't have full control over what would | |||
appear in PO files and you cannot use plural forms or context. In order to do | |||
that you have to use `i18n.inc` | |||
`i18n.inc` contains several macros that, when used, make a string translatable. | |||
The first thing to do is to make these macros available by adding | |||
[% PROCESS 'i18n.inc' %] | |||
at the top of the template file. Then you can use those macros. | |||
The simplest one is `t(msgid)` | |||
[% t('This is a translatable string') %] | |||
You can also use variable substitution with `tx(msgid, vars)` | |||
[% tx('Hello, {name}', { name = 'World' }) %] | |||
You can use plural forms with `tn(msgid, msgid_plural, count)` | |||
[% tn('a child', 'several children', number_of_children) %] | |||
You can add context, to help translators when a term is ambiguous, with | |||
`tp(msgctxt, msgid)` | |||
[% tp('verb', 'order') %] | |||
[% tp('noun', 'order') %] | |||
Or any combinations of the above | |||
[% tnpx('bibliographic record', '{count} item', '{count} items', items_count, { count = items_count }) %] | |||
### In JavaScript files (`*.js`) | |||
Like in templates, you have several functions available. Just replace `t` by `__`. | |||
__('This is a translatable string'); | |||
__npx('bibliographic record, '{count} item', '{count} items', items_count, { count: items_count }); | |||
### In Perl files (`*.pl`, `*.pm`) | |||
You will have to add | |||
use Koha::I18N; | |||
at the top of the file, and then the same functions as above will be available. | |||
__('This is a translatable string'); | |||
__npx('bibliographic record, '{count} item', '{count} items', $items_count, count => $items_count); | |||
### In installer and preferences YAML files (`*.yml`) | |||
Nothing special to do here. All strings will be automatically translatable. | |||
## Manipulating PO files | |||
Once strings have been made translatable in source files, they have to be | |||
extracted into PO files and uploaded on https://translate.koha-community.org/ | |||
so they can be translated. | |||
### Install gulp first | |||
The next sections rely on gulp. If it's not installed, run the following | |||
commands: | |||
# as root | |||
npm install gulp-cli -g | |||
# as normal user, from the root of Koha repository | |||
yarn | |||
### Create PO files for a new language | |||
If you want to add translations for a new language, you have to create the | |||
missing PO files. You can do that by executing the following command: | |||
# Replace xx-XX by your language tag | |||
gulp po:create --lang xx-XX | |||
New PO files will be available in `misc/translator/po`. | |||
### Update PO files with new strings | |||
When new features or bugfixes are added to Koha, new translatable strings can | |||
be added, other can be removed or modified, and the PO file become out of sync. | |||
To be able to translate the new or modified strings, you have to update PO | |||
files. This can be done by executing the following command: | |||
# Update PO files for all languages | |||
gulp po:update | |||
# or only one language | |||
gulp po:update --lang xx-XX | |||
### Only extract strings | |||
Creating or updating PO files automatically extract strings, but if for some | |||
reasons you want to only extract strings without touching PO files, you can run | |||
the following command: | |||
gulp po:extract | |||
POT files will be available in `misc/translator`. |
@@ -1,13 +1,24 @@ | |||
/* eslint-env node */ | |||
/* eslint no-console:"off" */ | |||
const { dest, series, src, watch } = require('gulp'); | |||
const { dest, parallel, series, src, watch } = require('gulp'); | |||
const child_process = require('child_process'); | |||
const fs = require('fs'); | |||
const os = require('os'); | |||
const path = require('path'); | |||
const util = require('util'); | |||
const sass = require("gulp-sass"); | |||
const cssnano = require("gulp-cssnano"); | |||
const rtlcss = require('gulp-rtlcss'); | |||
const sourcemaps = require('gulp-sourcemaps'); | |||
const autoprefixer = require('gulp-autoprefixer'); | |||
const concatPo = require('gulp-concat-po'); | |||
const exec = require('gulp-exec'); | |||
const merge = require('merge-stream'); | |||
const through2 = require('through2'); | |||
const Vinyl = require('vinyl'); | |||
const args = require('minimist')(process.argv.slice(2)); | |||
const rename = require('gulp-rename'); | |||
@@ -62,8 +73,297 @@ function build() { | |||
.pipe(dest(css_base)); | |||
} | |||
const poTasks = { | |||
'marc-MARC21': { | |||
extract: po_extract_marc_marc21, | |||
create: po_create_marc_marc21, | |||
update: po_update_marc_marc21, | |||
}, | |||
'marc-NORMARC': { | |||
extract: po_extract_marc_normarc, | |||
create: po_create_marc_normarc, | |||
update: po_update_marc_normarc, | |||
}, | |||
'marc-UNIMARC': { | |||
extract: po_extract_marc_unimarc, | |||
create: po_create_marc_unimarc, | |||
update: po_update_marc_unimarc, | |||
}, | |||
'staff-prog': { | |||
extract: po_extract_staff, | |||
create: po_create_staff, | |||
update: po_update_staff, | |||
}, | |||
'opac-bootstrap': { | |||
extract: po_extract_opac, | |||
create: po_create_opac, | |||
update: po_update_opac, | |||
}, | |||
'pref': { | |||
extract: po_extract_pref, | |||
create: po_create_pref, | |||
update: po_update_pref, | |||
}, | |||
'messages': { | |||
extract: po_extract_messages, | |||
create: po_create_messages, | |||
update: po_update_messages, | |||
}, | |||
'messages-js': { | |||
extract: po_extract_messages_js, | |||
create: po_create_messages_js, | |||
update: po_update_messages_js, | |||
}, | |||
'installer': { | |||
extract: po_extract_installer, | |||
create: po_create_installer, | |||
update: po_update_installer, | |||
}, | |||
'installer-MARC21': { | |||
extract: po_extract_installer_marc21, | |||
create: po_create_installer_marc21, | |||
update: po_update_installer_marc21, | |||
}, | |||
}; | |||
const poTypes = Object.keys(poTasks); | |||
function po_extract_marc (type) { | |||
return src(`koha-tmpl/*-tmpl/*/en/**/*${type}*`, { read: false, nocase: true }) | |||
.pipe(xgettext('misc/translator/xgettext.pl --charset=UTF-8 -s', `Koha-marc-${type}.pot`)) | |||
.pipe(dest('misc/translator')) | |||
} | |||
function po_extract_marc_marc21 () { return po_extract_marc('MARC21') } | |||
function po_extract_marc_normarc () { return po_extract_marc('NORMARC') } | |||
function po_extract_marc_unimarc () { return po_extract_marc('UNIMARC') } | |||
function po_extract_staff () { | |||
const globs = [ | |||
'koha-tmpl/intranet-tmpl/prog/en/**/*.tt', | |||
'koha-tmpl/intranet-tmpl/prog/en/**/*.inc', | |||
'koha-tmpl/intranet-tmpl/prog/en/xslt/*.xsl', | |||
'koha-tmpl/intranet-tmpl/prog/en/columns.def', | |||
'!koha-tmpl/intranet-tmpl/prog/en/**/*MARC21*', | |||
'!koha-tmpl/intranet-tmpl/prog/en/**/*NORMARC*', | |||
'!koha-tmpl/intranet-tmpl/prog/en/**/*UNIMARC*', | |||
'!koha-tmpl/intranet-tmpl/prog/en/**/*marc21*', | |||
'!koha-tmpl/intranet-tmpl/prog/en/**/*normarc*', | |||
'!koha-tmpl/intranet-tmpl/prog/en/**/*unimarc*', | |||
]; | |||
return src(globs, { read: false, nocase: true }) | |||
.pipe(xgettext('misc/translator/xgettext.pl --charset=UTF-8 -s', 'Koha-staff-prog.pot')) | |||
.pipe(dest('misc/translator')) | |||
} | |||
function po_extract_opac () { | |||
const globs = [ | |||
'koha-tmpl/opac-tmpl/bootstrap/en/**/*.tt', | |||
'koha-tmpl/opac-tmpl/bootstrap/en/**/*.inc', | |||
'koha-tmpl/opac-tmpl/bootstrap/en/xslt/*.xsl', | |||
'!koha-tmpl/opac-tmpl/bootstrap/en/**/*MARC21*', | |||
'!koha-tmpl/opac-tmpl/bootstrap/en/**/*NORMARC*', | |||
'!koha-tmpl/opac-tmpl/bootstrap/en/**/*UNIMARC*', | |||
'!koha-tmpl/opac-tmpl/bootstrap/en/**/*marc21*', | |||
'!koha-tmpl/opac-tmpl/bootstrap/en/**/*normarc*', | |||
'!koha-tmpl/opac-tmpl/bootstrap/en/**/*unimarc*', | |||
]; | |||
return src(globs, { read: false, nocase: true }) | |||
.pipe(xgettext('misc/translator/xgettext.pl --charset=UTF-8 -s', 'Koha-opac-bootstrap.pot')) | |||
.pipe(dest('misc/translator')) | |||
} | |||
const xgettext_options = '--from-code=UTF-8 --package-name Koha ' | |||
+ '--package-version= -k -k__ -k__x -k__n:1,2 -k__nx:1,2 -k__xn:1,2 ' | |||
+ '-k__p:1c,2 -k__px:1c,2 -k__np:1c,2,3 -k__npx:1c,2,3 -kN__ ' | |||
+ '-kN__n:1,2 -kN__p:1c,2 -kN__np:1c,2,3 --force-po'; | |||
function po_extract_messages_js () { | |||
const globs = [ | |||
'koha-tmpl/intranet-tmpl/prog/js/**/*.js', | |||
'koha-tmpl/opac-tmpl/bootstrap/js/**/*.js', | |||
]; | |||
return src(globs, { read: false, nocase: true }) | |||
.pipe(xgettext(`xgettext -L JavaScript ${xgettext_options}`, 'Koha-messages-js.pot')) | |||
.pipe(dest('misc/translator')) | |||
} | |||
function po_extract_messages () { | |||
const perlStream = src(['**/*.pl', '**/*.pm'], { read: false, nocase: true }) | |||
.pipe(xgettext(`xgettext -L Perl ${xgettext_options}`, 'Koha-perl.pot')) | |||
const ttStream = src([ | |||
'koha-tmpl/intranet-tmpl/prog/en/**/*.tt', | |||
'koha-tmpl/intranet-tmpl/prog/en/**/*.inc', | |||
'koha-tmpl/opac-tmpl/bootstrap/en/**/*.tt', | |||
'koha-tmpl/opac-tmpl/bootstrap/en/**/*.inc', | |||
], { read: false, nocase: true }) | |||
.pipe(xgettext('misc/translator/xgettext-tt2 --from-code=UTF-8', 'Koha-tt.pot')) | |||
const headers = { | |||
'Project-Id-Version': 'Koha', | |||
'Content-Type': 'text/plain; charset=UTF-8', | |||
}; | |||
return merge(perlStream, ttStream) | |||
.pipe(concatPo('Koha-messages.pot', { headers })) | |||
.pipe(dest('misc/translator')) | |||
} | |||
function po_extract_pref () { | |||
return src('koha-tmpl/intranet-tmpl/prog/en/modules/admin/preferences/*.pref', { read: false }) | |||
.pipe(xgettext('misc/translator/xgettext-pref', 'Koha-pref.pot')) | |||
.pipe(dest('misc/translator')) | |||
} | |||
function po_extract_installer () { | |||
const globs = [ | |||
'installer/data/mysql/en/mandatory/*.yml', | |||
'installer/data/mysql/en/optional/*.yml', | |||
]; | |||
return src(globs, { read: false, nocase: true }) | |||
.pipe(xgettext('misc/translator/xgettext-installer', 'Koha-installer.pot')) | |||
.pipe(dest('misc/translator')) | |||
} | |||
function po_extract_installer_marc (type) { | |||
const globs = `installer/data/mysql/en/marcflavour/${type}/**/*.yml`; | |||
return src(globs, { read: false, nocase: true }) | |||
.pipe(xgettext('misc/translator/xgettext-installer', `Koha-installer-${type}.pot`)) | |||
.pipe(dest('misc/translator')) | |||
} | |||
function po_extract_installer_marc21 () { return po_extract_installer_marc('MARC21') } | |||
function po_create_type (type) { | |||
const access = util.promisify(fs.access); | |||
const exec = util.promisify(child_process.exec); | |||
const languages = getLanguages(); | |||
const promises = []; | |||
for (const language of languages) { | |||
const locale = language.split('-').filter(s => s.length !== 4).join('_'); | |||
const po = `misc/translator/po/${language}-${type}.po`; | |||
const pot = `misc/translator/Koha-${type}.pot`; | |||
const promise = access(po) | |||
.catch(() => exec(`msginit -o ${po} -i ${pot} -l ${locale} --no-translator`)) | |||
promises.push(promise); | |||
} | |||
return Promise.all(promises); | |||
} | |||
function po_create_marc_marc21 () { return po_create_type('marc-MARC21') } | |||
function po_create_marc_normarc () { return po_create_type('marc-NORMARC') } | |||
function po_create_marc_unimarc () { return po_create_type('marc-UNIMARC') } | |||
function po_create_staff () { return po_create_type('staff-prog') } | |||
function po_create_opac () { return po_create_type('opac-bootstrap') } | |||
function po_create_pref () { return po_create_type('pref') } | |||
function po_create_messages () { return po_create_type('messages') } | |||
function po_create_messages_js () { return po_create_type('messages-js') } | |||
function po_create_installer () { return po_create_type('installer') } | |||
function po_create_installer_marc21 () { return po_create_type('installer-MARC21') } | |||
function po_update_type (type) { | |||
const msgmerge_opts = '--backup=off --quiet --sort-output --update'; | |||
const cmd = `msgmerge ${msgmerge_opts} <%= file.path %> misc/translator/Koha-${type}.pot`; | |||
const languages = getLanguages(); | |||
const globs = languages.map(language => `misc/translator/po/${language}-${type}.po`); | |||
return src(globs) | |||
.pipe(exec(cmd, { continueOnError: true })) | |||
.pipe(exec.reporter({ err: false, stdout: false })) | |||
} | |||
function po_update_marc_marc21 () { return po_update_type('marc-MARC21') } | |||
function po_update_marc_normarc () { return po_update_type('marc-NORMARC') } | |||
function po_update_marc_unimarc () { return po_update_type('marc-UNIMARC') } | |||
function po_update_staff () { return po_update_type('staff-prog') } | |||
function po_update_opac () { return po_update_type('opac-bootstrap') } | |||
function po_update_pref () { return po_update_type('pref') } | |||
function po_update_messages () { return po_update_type('messages') } | |||
function po_update_messages_js () { return po_update_type('messages-js') } | |||
function po_update_installer () { return po_update_type('installer') } | |||
function po_update_installer_marc21 () { return po_update_type('installer-MARC21') } | |||
/** | |||
* Gulp plugin that executes xgettext-like command `cmd` on all files given as | |||
* input, and then outputs the result as a POT file named `filename`. | |||
* `cmd` should accept -o and -f options | |||
*/ | |||
function xgettext (cmd, filename) { | |||
const filenames = []; | |||
function transform (file, encoding, callback) { | |||
filenames.push(path.relative(file.cwd, file.path)); | |||
callback(); | |||
} | |||
function flush (callback) { | |||
fs.mkdtemp(path.join(os.tmpdir(), 'koha-'), (err, folder) => { | |||
const outputFilename = path.join(folder, filename); | |||
const filesFilename = path.join(folder, 'files'); | |||
fs.writeFile(filesFilename, filenames.join(os.EOL), err => { | |||
if (err) return callback(err); | |||
const command = `${cmd} -o ${outputFilename} -f ${filesFilename}`; | |||
child_process.exec(command, err => { | |||
if (err) return callback(err); | |||
fs.readFile(outputFilename, (err, data) => { | |||
if (err) return callback(err); | |||
const file = new Vinyl(); | |||
file.path = path.join(file.base, filename); | |||
file.contents = data; | |||
callback(null, file); | |||
}); | |||
}); | |||
}); | |||
}) | |||
} | |||
return through2.obj(transform, flush); | |||
} | |||
/** | |||
* Return languages selected for PO-related tasks | |||
* | |||
* This can be either languages given on command-line with --lang option, or | |||
* all the languages found in misc/translator/po otherwise | |||
*/ | |||
function getLanguages () { | |||
if (Array.isArray(args.lang)) { | |||
return args.lang; | |||
} | |||
if (args.lang) { | |||
return [args.lang]; | |||
} | |||
const filenames = fs.readdirSync('misc/translator/po') | |||
.filter(filename => filename.endsWith('.po')) | |||
.filter(filename => !filename.startsWith('.')) | |||
const re = new RegExp('-(' + poTypes.join('|') + ')\.po$'); | |||
languages = filenames.map(filename => filename.replace(re, '')) | |||
return Array.from(new Set(languages)); | |||
} | |||
exports.build = build; | |||
exports.css = css; | |||
exports['po:create'] = parallel(...poTypes.map(type => series(poTasks[type].extract, poTasks[type].create))); | |||
exports['po:update'] = parallel(...poTypes.map(type => series(poTasks[type].extract, poTasks[type].update))); | |||
exports['po:extract'] = parallel(...poTypes.map(type => poTasks[type].extract)); | |||
exports.default = function () { | |||
watch(css_base + "/src/**/*.scss", series('css')); | |||
} |
@@ -22,36 +22,15 @@ use Modern::Perl; | |||
use C4::Context; | |||
# WARNING: Any other tested YAML library fails to work properly in this | |||
# script content | |||
use YAML::Syck qw( Dump LoadFile DumpFile ); | |||
use YAML::Syck qw( LoadFile DumpFile ); | |||
use Locale::PO; | |||
use FindBin qw( $Bin ); | |||
use File::Basename; | |||
use File::Find; | |||
use File::Path qw( make_path ); | |||
use File::Copy; | |||
use File::Slurp; | |||
use File::Spec; | |||
use File::Temp qw( tempdir tempfile ); | |||
use Template::Parser; | |||
use PPI; | |||
$YAML::Syck::ImplicitTyping = 1; | |||
# Default file header for .po syspref files | |||
my $default_pref_po_header = Locale::PO->new(-msgid => '', -msgstr => | |||
"Project-Id-Version: PACKAGE VERSION\\n" . | |||
"PO-Revision-Date: YEAR-MO-DA HO:MI +ZONE\\n" . | |||
"Last-Translator: FULL NAME <EMAIL\@ADDRESS>\\n" . | |||
"Language-Team: Koha Translate List <koha-translate\@lists.koha-community.org>\\n" . | |||
"MIME-Version: 1.0\\n" . | |||
"Content-Type: text/plain; charset=UTF-8\\n" . | |||
"Content-Transfer-Encoding: 8bit\\n" . | |||
"Plural-Forms: nplurals=2; plural=(n > 1);\\n" | |||
); | |||
sub set_lang { | |||
my ($self, $lang) = @_; | |||
@@ -60,7 +39,6 @@ sub set_lang { | |||
"/prog/$lang/modules/admin/preferences"; | |||
} | |||
sub new { | |||
my ($class, $lang, $pref_only, $verbose) = @_; | |||
@@ -75,32 +53,16 @@ sub new { | |||
$self->{verbose} = $verbose; | |||
$self->{process} = "$Bin/tmpl_process3.pl " . ($verbose ? '' : '-q'); | |||
$self->{path_po} = "$Bin/po"; | |||
$self->{po} = { '' => $default_pref_po_header }; | |||
$self->{po} = {}; | |||
$self->{domain} = 'Koha'; | |||
$self->{cp} = `which cp`; | |||
$self->{msgmerge} = `which msgmerge`; | |||
$self->{msgfmt} = `which msgfmt`; | |||
$self->{msginit} = `which msginit`; | |||
$self->{msgattrib} = `which msgattrib`; | |||
$self->{xgettext} = `which xgettext`; | |||
$self->{sed} = `which sed`; | |||
$self->{po2json} = "$Bin/po2json"; | |||
$self->{gzip} = `which gzip`; | |||
$self->{gunzip} = `which gunzip`; | |||
chomp $self->{cp}; | |||
chomp $self->{msgmerge}; | |||
chomp $self->{msgfmt}; | |||
chomp $self->{msginit}; | |||
chomp $self->{msgattrib}; | |||
chomp $self->{xgettext}; | |||
chomp $self->{sed}; | |||
chomp $self->{gzip}; | |||
chomp $self->{gunzip}; | |||
unless ($self->{xgettext}) { | |||
die "Missing 'xgettext' executable. Have you installed the gettext package?\n"; | |||
} | |||
# Get all .pref file names | |||
opendir my $fh, $self->{path_pref_en}; | |||
my @pref_files = grep { /\.pref$/ } readdir($fh); | |||
@@ -175,7 +137,6 @@ sub new { | |||
bless $self, $class; | |||
} | |||
sub po_filename { | |||
my $self = shift; | |||
my $suffix = shift; | |||
@@ -186,162 +147,92 @@ sub po_filename { | |||
return $trans_file; | |||
} | |||
sub get_trans_text { | |||
my ($self, $msgid, $default) = @_; | |||
sub po_append { | |||
my ($self, $id, $comment) = @_; | |||
my $po = $self->{po}; | |||
my $p = $po->{$id}; | |||
if ( $p ) { | |||
$p->comment( $p->comment . "\n" . $comment ); | |||
} | |||
else { | |||
$po->{$id} = Locale::PO->new( | |||
-comment => $comment, | |||
-msgid => $id, | |||
-msgstr => '' | |||
); | |||
} | |||
} | |||
sub add_prefs { | |||
my ($self, $comment, $prefs) = @_; | |||
for my $pref ( @$prefs ) { | |||
my $pref_name = ''; | |||
for my $element ( @$pref ) { | |||
if ( ref( $element) eq 'HASH' ) { | |||
$pref_name = $element->{pref}; | |||
last; | |||
} | |||
} | |||
for my $element ( @$pref ) { | |||
if ( ref( $element) eq 'HASH' ) { | |||
while ( my ($key, $value) = each(%$element) ) { | |||
next unless $key eq 'choices' or $key eq 'multiple'; | |||
next unless ref($value) eq 'HASH'; | |||
for my $ckey ( keys %$value ) { | |||
my $id = $self->{file} . "#$pref_name# " . $value->{$ckey}; | |||
$self->po_append( $id, $comment ); | |||
} | |||
} | |||
} | |||
elsif ( $element ) { | |||
$self->po_append( $self->{file} . "#$pref_name# $element", $comment ); | |||
} | |||
my $po = $self->{po}->{Locale::PO->quote($msgid)}; | |||
if ($po) { | |||
my $msgstr = Locale::PO->dequote($po->msgstr); | |||
if ($msgstr and length($msgstr) > 0) { | |||
return $msgstr; | |||
} | |||
} | |||
} | |||
sub get_trans_text { | |||
my ($self, $id) = @_; | |||
my $po = $self->{po}->{$id}; | |||
return unless $po; | |||
return Locale::PO->dequote($po->msgstr); | |||
return $default; | |||
} | |||
sub get_translated_tab_content { | |||
my ($self, $file, $tab_content) = @_; | |||
sub update_tab_prefs { | |||
my ($self, $pref, $prefs) = @_; | |||
for my $p ( @$prefs ) { | |||
my $pref_name = ''; | |||
next unless $p; | |||
for my $element ( @$p ) { | |||
if ( ref( $element) eq 'HASH' ) { | |||
$pref_name = $element->{pref}; | |||
last; | |||
} | |||
} | |||
for my $i ( 0..@$p-1 ) { | |||
my $element = $p->[$i]; | |||
if ( ref( $element) eq 'HASH' ) { | |||
while ( my ($key, $value) = each(%$element) ) { | |||
next unless $key eq 'choices' or $key eq 'multiple'; | |||
next unless ref($value) eq 'HASH'; | |||
for my $ckey ( keys %$value ) { | |||
my $id = $self->{file} . "#$pref_name# " . $value->{$ckey}; | |||
my $text = $self->get_trans_text( $id ); | |||
$value->{$ckey} = $text if $text; | |||
} | |||
} | |||
} | |||
elsif ( $element ) { | |||
my $id = $self->{file} . "#$pref_name# $element"; | |||
my $text = $self->get_trans_text( $id ); | |||
$p->[$i] = $text if $text; | |||
} | |||
} | |||
if ( ref($tab_content) eq 'ARRAY' ) { | |||
return $self->get_translated_prefs($file, $tab_content); | |||
} | |||
} | |||
my $translated_tab_content = { | |||
map { | |||
my $section = $_; | |||
my $sysprefs = $tab_content->{$section}; | |||
my $msgid = sprintf('%s %s', $file, $section); | |||
sub get_po_from_prefs { | |||
my $self = shift; | |||
$self->get_trans_text($msgid, $section) => $self->get_translated_prefs($file, $sysprefs); | |||
} keys %$tab_content | |||
}; | |||
for my $file ( @{$self->{pref_files}} ) { | |||
my $pref = LoadFile( $self->{path_pref_en} . "/$file" ); | |||
$self->{file} = $file; | |||
# Entries for tab titles | |||
$self->po_append( $self->{file}, $_ ) for keys %$pref; | |||
while ( my ($tab, $tab_content) = each %$pref ) { | |||
if ( ref($tab_content) eq 'ARRAY' ) { | |||
$self->add_prefs( $tab, $tab_content ); | |||
next; | |||
} | |||
while ( my ($section, $sysprefs) = each %$tab_content ) { | |||
my $comment = "$tab > $section"; | |||
$self->po_append( $self->{file} . " " . $section, $comment ); | |||
$self->add_prefs( $comment, $sysprefs ); | |||
} | |||
} | |||
} | |||
return $translated_tab_content; | |||
} | |||
sub get_translated_prefs { | |||
my ($self, $file, $sysprefs) = @_; | |||
sub save_po { | |||
my $self = shift; | |||
my $translated_prefs = [ | |||
map { | |||
my ($pref_elt) = grep { ref($_) eq 'HASH' && exists $_->{pref} } @$_; | |||
my $pref_name = $pref_elt ? $pref_elt->{pref} : ''; | |||
my $translated_syspref = [ | |||
map { | |||
$self->get_translated_pref($file, $pref_name, $_); | |||
} @$_ | |||
]; | |||
# Create file header if it doesn't already exist | |||
my $po = $self->{po}; | |||
$po->{''} ||= $default_pref_po_header; | |||
$translated_syspref; | |||
} @$sysprefs | |||
]; | |||
# Write .po entries into a file put in Koha standard po directory | |||
Locale::PO->save_file_fromhash( $self->po_filename("-pref.po"), $po ); | |||
say "Saved in file: ", $self->po_filename("-pref.po") if $self->{verbose}; | |||
return $translated_prefs; | |||
} | |||
sub get_translated_pref { | |||
my ($self, $file, $pref_name, $syspref) = @_; | |||
sub get_po_merged_with_en { | |||
my $self = shift; | |||
# Get po from current 'en' .pref files | |||
$self->get_po_from_prefs(); | |||
my $po_current = $self->{po}; | |||
unless (ref($syspref)) { | |||
$syspref //= ''; | |||
my $msgid = sprintf('%s#%s# %s', $file, $pref_name, $syspref); | |||
return $self->get_trans_text($msgid, $syspref); | |||
} | |||
# Get po from previous generation | |||
my $po_previous = Locale::PO->load_file_ashash( $self->po_filename("-pref.po") ); | |||
my $translated_pref = { | |||
map { | |||
my $key = $_; | |||
my $value = $syspref->{$key}; | |||
for my $id ( keys %$po_current ) { | |||
my $po = $po_previous->{Locale::PO->quote($id)}; | |||
next unless $po; | |||
my $text = Locale::PO->dequote( $po->msgstr ); | |||
$po_current->{$id}->msgstr( $text ); | |||
} | |||
} | |||
my $translated_value = $value; | |||
if (($key eq 'choices' || $key eq 'multiple') && ref($value) eq 'HASH') { | |||
$translated_value = { | |||
map { | |||
my $msgid = sprintf('%s#%s# %s', $file, $pref_name, $value->{$_}); | |||
$_ => $self->get_trans_text($msgid, $value->{$_}) | |||
} keys %$value | |||
} | |||
} | |||
$key => $translated_value | |||
} keys %$syspref | |||
}; | |||
sub update_prefs { | |||
my $self = shift; | |||
print "Update '", $self->{lang}, | |||
"' preferences .po file from 'en' .pref files\n" if $self->{verbose}; | |||
$self->get_po_merged_with_en(); | |||
$self->save_po(); | |||
return $translated_pref; | |||
} | |||
sub install_prefs { | |||
my $self = shift; | |||
@@ -350,45 +241,24 @@ sub install_prefs { | |||
exit; | |||
} | |||
# Get the language .po file merged with last modified 'en' preferences | |||
$self->get_po_merged_with_en(); | |||
$self->{po} = Locale::PO->load_file_ashash($self->po_filename("-pref.po"), 'utf8'); | |||
for my $file ( @{$self->{pref_files}} ) { | |||
my $pref = LoadFile( $self->{path_pref_en} . "/$file" ); | |||
$self->{file} = $file; | |||
# First, keys are replaced (tab titles) | |||
$pref = do { | |||
my %pref = map { | |||
$self->get_trans_text( $self->{file} ) || $_ => $pref->{$_} | |||
} keys %$pref; | |||
\%pref; | |||
my $translated_pref = { | |||
map { | |||
my $tab = $_; | |||
my $tab_content = $pref->{$tab}; | |||
$self->get_trans_text($file, $tab) => $self->get_translated_tab_content($file, $tab_content); | |||
} keys %$pref | |||
}; | |||
while ( my ($tab, $tab_content) = each %$pref ) { | |||
if ( ref($tab_content) eq 'ARRAY' ) { | |||
$self->update_tab_prefs( $pref, $tab_content ); | |||
next; | |||
} | |||
while ( my ($section, $sysprefs) = each %$tab_content ) { | |||
$self->update_tab_prefs( $pref, $sysprefs ); | |||
} | |||
my $ntab = {}; | |||
for my $section ( keys %$tab_content ) { | |||
my $id = $self->{file} . " $section"; | |||
my $text = $self->get_trans_text($id); | |||
my $nsection = $text ? $text : $section; | |||
if( exists $ntab->{$nsection} ) { | |||
# When translations collide (see BZ 18634) | |||
push @{$ntab->{$nsection}}, @{$tab_content->{$section}}; | |||
} else { | |||
$ntab->{$nsection} = $tab_content->{$section}; | |||
} | |||
} | |||
$pref->{$tab} = $ntab; | |||
} | |||
my $file_trans = $self->{po_path_lang} . "/$file"; | |||
print "Write $file\n" if $self->{verbose}; | |||
open my $fh, ">", $file_trans; | |||
print $fh Dump($pref); | |||
DumpFile($file_trans, $translated_pref); | |||
} | |||
} | |||
@@ -429,180 +299,6 @@ sub install_tmpl { | |||
} | |||
} | |||
sub update_tmpl { | |||
my ($self, $files) = @_; | |||
say "Update templates" if $self->{verbose}; | |||
for my $trans ( @{$self->{interface}} ) { | |||
my @files = @$files; | |||
my @nomarc = (); | |||
" Update templates '$trans->{name}'\n", | |||
" From: $trans->{dir}/en/\n", | |||
" To : $self->{path_po}/$self->{lang}$trans->{suffix}\n" | |||
if $self->{verbose}; | |||
my $trans_dir = join("/en/ -i ",split(" ",$trans->{dir}))."/en/"; # multiple source dirs | |||
# if processing MARC po file, only use corresponding files | |||
my $marc = ( $trans->{name} =~ /MARC/ )?"-m \"$trans->{name}\"":""; # for MARC translations | |||
# if not processing MARC po file, ignore all MARC files | |||
@nomarc = ( 'marc21', 'unimarc', 'normarc' ) if ( $trans->{name} !~ /MARC/ ); # hardcoded MARC variants | |||
system | |||
"$self->{process} update " . | |||
"-i $trans_dir " . | |||
"-s $self->{path_po}/$self->{lang}$trans->{suffix} -r " . | |||
"$marc " . | |||
( @files ? ' -f ' . join ' -f ', @files : '') . | |||
( @nomarc ? ' -n ' . join ' -n ', @nomarc : ''); | |||
} | |||
} | |||
sub create_prefs { | |||
my $self = shift; | |||
if ( -e $self->po_filename("-pref.po") ) { | |||
say "Preferences .po file already exists. Delete it if you want to recreate it."; | |||
return; | |||
} | |||
$self->get_po_from_prefs(); | |||
$self->save_po(); | |||
} | |||
sub get_po_from_target { | |||
my $self = shift; | |||
my $target = shift; | |||
my $po; | |||
my $po_head = Locale::PO->new; | |||
$po_head->{msgid} = "\"\""; | |||
$po_head->{msgstr} = "". | |||
"Project-Id-Version: Koha Project - Installation files\\n" . | |||
"PO-Revision-Date: YEAR-MO-DA HO:MI +ZONE\\n" . | |||
"Last-Translator: FULL NAME <EMAIL\@ADDRESS>\\n" . | |||
"Language-Team: Koha Translation Team\\n" . | |||
"Language: ".$self->{lang}."\\n" . | |||
"MIME-Version: 1.0\\n" . | |||
"Content-Type: text/plain; charset=UTF-8\\n" . | |||
"Content-Transfer-Encoding: 8bit\\n"; | |||
my @dirs = @{ $target->{dirs} }; | |||
my $intradir = $self->{context}->config('intranetdir'); | |||
for my $dir ( @dirs ) { # each dir | |||
opendir( my $dh, "$intradir/$dir" ) or die ("Can't open $intradir/$dir"); | |||
my @filelist = grep { $_ =~ m/\.yml/ } readdir($dh); # Just yaml files | |||
close($dh); | |||
for my $file ( @filelist ) { # each file | |||
my $yaml = LoadFile( "$intradir/$dir/$file" ); | |||
my @tables = @{ $yaml->{'tables'} }; | |||
my $tablec; | |||
for my $table ( @tables ) { # each table | |||
$tablec++; | |||
my $table_name = ( keys %$table )[0]; | |||
my @translatable = @{ $table->{$table_name}->{translatable} }; | |||
my @rows = @{ $table->{$table_name}->{rows} }; | |||
my @multiline = @{ $table->{$table_name}->{'multiline'} }; # to check multiline values | |||
my $rowc; | |||
for my $row ( @rows ) { # each row | |||
$rowc++; | |||
for my $field ( @translatable ) { # each field | |||
if ( @multiline and grep { $_ eq $field } @multiline ) { # multiline fields, only notices ATM | |||
my $mulc; | |||
foreach my $line ( @{$row->{$field}} ) { | |||
$mulc++; | |||
next if ( $line =~ /^(\s*<.*?>\s*$|^\s*\[.*?\]\s*|\s*)$/ ); # discard pure html, TT, empty | |||
$line =~ s/(<<.*?>>|\[\%.*?\%\]|<.*?>)/\%s/g; # put placeholders | |||
next if ( $line =~ /^(\s|%s|-|[[:punct:]]|\(|\))*$/ or length($line) < 2 ); # discard non strings | |||
if ( not $po->{ $line } ) { | |||
my $msg = Locale::PO->new( | |||
-msgid => $line, -msgstr => '', | |||
-reference => "$dir/$file:$table_name:$tablec:row:$rowc:mul:$mulc" ); | |||
$po->{ $line } = $msg; | |||
} | |||
} | |||
} else { | |||
if ( defined $row->{$field} and length($row->{$field}) > 1 # discard null values and small strings | |||
and not $po->{ $row->{$field} } ) { | |||
my $msg = Locale::PO->new( | |||
-msgid => $row->{$field}, -msgstr => '', | |||
-reference => "$dir/$file:$table_name:$tablec:row:$rowc" ); | |||
$po->{ $row->{$field} } = $msg; | |||
} | |||
} | |||
} | |||
} | |||
} | |||
my $desccount; | |||
for my $description ( @{ $yaml->{'description'} } ) { | |||
$desccount++; | |||
if ( length($description) > 1 and not $po->{ $description } ) { | |||
my $msg = Locale::PO->new( | |||
-msgid => $description, -msgstr => '', | |||
-reference => "$dir/$file:description:$desccount" ); | |||
$po->{ $description } = $msg; | |||
} | |||
} | |||
} | |||
} | |||
$po->{''} = $po_head if ( $po ); | |||
return $po; | |||
} | |||
sub create_installer { | |||
my $self = shift; | |||
return unless ( $self->{installer} ); | |||
say "Create installer translation files\n" if $self->{verbose}; | |||
my @targets = @{ $self->{installer} }; # each installer target (common,marc21,unimarc) | |||
for my $target ( @targets ) { | |||
if ( -e $self->po_filename( $target->{suffix} ) ) { | |||
say "$self->{lang}$target->{suffix} file already exists. Delete it if you want to recreate it."; | |||
return; | |||
} | |||
} | |||
for my $target ( @targets ) { | |||
my $po = get_po_from_target( $self, $target ); | |||
# create output file only if there is something to write | |||
if ( $po ) { | |||
my $po_file = $self->po_filename( $target->{suffix} ); | |||
Locale::PO->save_file_fromhash( $po_file, $po ); | |||
say "Saved in file: ", $po_file if $self->{verbose}; | |||
} | |||
} | |||
} | |||
sub update_installer { | |||
my $self = shift; | |||
return unless ( $self->{installer} ); | |||
say "Update installer translation files\n" if $self->{verbose}; | |||
my @targets = @{ $self->{installer} }; # each installer target (common,marc21,unimarc) | |||
for my $target ( @targets ) { | |||
return unless ( -e $self->po_filename( $target->{suffix} ) ); | |||
my $po = get_po_from_target( $self, $target ); | |||
# update file only if there is something to update | |||
if ( $po ) { | |||
my ( $fh, $po_temp ) = tempfile(); | |||
binmode( $fh, ":encoding(UTF-8)" ); | |||
Locale::PO->save_file_fromhash( $po_temp, $po ); | |||
my $po_file = $self->po_filename( $target->{suffix} ); | |||
eval { | |||
my $st = system($self->{msgmerge}." ".($self->{verbose}?'':'-q'). | |||
" -s $po_file $po_temp -o - | ".$self->{msgattrib}." --no-obsolete -o $po_file"); | |||
}; | |||
say "Updated file: ", $po_file if $self->{verbose}; | |||
} | |||
} | |||
} | |||
sub translate_yaml { | |||
my $self = shift; | |||
my $target = shift; | |||
@@ -716,35 +412,6 @@ sub install_installer { | |||
} | |||
} | |||
sub create_tmpl { | |||
my ($self, $files) = @_; | |||
say "Create templates\n" if $self->{verbose}; | |||
for my $trans ( @{$self->{interface}} ) { | |||
my @files = @$files; | |||
my @nomarc = (); | |||
" Create templates .po files for '$trans->{name}'\n", | |||
" From: $trans->{dir}/en/\n", | |||
" To : $self->{path_po}/$self->{lang}$trans->{suffix}\n" | |||
if $self->{verbose}; | |||
my $trans_dir = join("/en/ -i ",split(" ",$trans->{dir}))."/en/"; # multiple source dirs | |||
# if processing MARC po file, only use corresponding files | |||
my $marc = ( $trans->{name} =~ /MARC/ )?"-m \"$trans->{name}\"":""; # for MARC translations | |||
# if not processing MARC po file, ignore all MARC files | |||
@nomarc = ( 'marc21', 'unimarc', 'normarc' ) if ( $trans->{name} !~ /MARC/ ); # hardcoded MARC variants | |||
system | |||
"$self->{process} create " . | |||
"-i $trans_dir " . | |||
"-s $self->{path_po}/$self->{lang}$trans->{suffix} -r " . | |||
"$marc " . | |||
( @files ? ' -f ' . join ' -f ', @files : '') . | |||
( @nomarc ? ' -n ' . join ' -n ', @nomarc : ''); | |||
} | |||
} | |||
sub locale_name { | |||
my $self = shift; | |||
@@ -758,250 +425,6 @@ sub locale_name { | |||
return $locale; | |||
} | |||
sub create_messages { | |||
my $self = shift; | |||
my $pot = "$Bin/$self->{domain}.pot"; | |||
my $po = "$self->{path_po}/$self->{lang}-messages.po"; | |||
my $js_pot = "$self->{domain}-js.pot"; | |||
my $js_po = "$self->{path_po}/$self->{lang}-messages-js.po"; | |||
unless ( -f $pot && -f $js_pot ) { | |||
$self->extract_messages(); | |||
} | |||
say "Create messages ($self->{lang})" if $self->{verbose}; | |||
my $locale = $self->locale_name(); | |||
system "$self->{msginit} -i $pot -o $po -l $locale --no-translator 2> /dev/null"; | |||
warn "Problems creating $pot ".$? if ( $? == -1 ); | |||
system "$self->{msginit} -i $js_pot -o $js_po -l $locale --no-translator 2> /dev/null"; | |||
warn "Problems creating $js_pot ".$? if ( $? == -1 ); | |||
# If msginit failed to correctly set Plural-Forms, set a default one | |||
system "$self->{sed} --in-place " | |||
. "--expression='s/Plural-Forms: nplurals=INTEGER; plural=EXPRESSION/Plural-Forms: nplurals=2; plural=(n != 1)/' " | |||
. "$po $js_po"; | |||
} | |||
sub update_messages { | |||
my $self = shift; | |||
my $pot = "$Bin/$self->{domain}.pot"; | |||
my $po = "$self->{path_po}/$self->{lang}-messages.po"; | |||
my $js_pot = "$self->{domain}-js.pot"; | |||
my $js_po = "$self->{path_po}/$self->{lang}-messages-js.po"; | |||
unless ( -f $pot && -f $js_pot ) { | |||
$self->extract_messages(); | |||
} | |||
if ( -f $po && -f $js_pot ) { | |||
say "Update messages ($self->{lang})" if $self->{verbose}; | |||
system "$self->{msgmerge} --backup=off --quiet -U $po $pot"; | |||
system "$self->{msgmerge} --backup=off --quiet -U $js_po $js_pot"; | |||
} else { | |||
$self->create_messages(); | |||
} | |||
} | |||
sub extract_messages_from_templates { | |||
my ($self, $tempdir, $type, @files) = @_; | |||
my $htdocs = $type eq 'intranet' ? 'intrahtdocs' : 'opachtdocs'; | |||
my $dir = $self->{context}->config($htdocs); | |||
my @keywords = qw(t tx tn txn tnx tp tpx tnp tnpx); | |||
my $parser = Template::Parser->new(); | |||
foreach my $file (@files) { | |||
say "Extract messages from $file" if $self->{verbose}; | |||
my $template = read_file(File::Spec->catfile($dir, $file)); | |||
# No need to process a file that doesn't use the i18n.inc file. | |||
next unless $template =~ /i18n\.inc/; | |||
my $data = $parser->parse($template); | |||
unless ($data) { | |||
warn "Error at $file : " . $parser->error(); | |||
next; | |||
} | |||
my $destfile = $type eq 'intranet' ? | |||
File::Spec->catfile($tempdir, 'koha-tmpl', 'intranet-tmpl', $file) : | |||
File::Spec->catfile($tempdir, 'koha-tmpl', 'opac-tmpl', $file); | |||
make_path(dirname($destfile)); | |||
open my $fh, '>', $destfile; | |||
my @blocks = ($data->{BLOCK}, values %{ $data->{DEFBLOCKS} }); | |||
foreach my $block (@blocks) { | |||
my $document = PPI::Document->new(\$block); | |||
# [% t('foo') %] is compiled to | |||
# $output .= $stash->get(['t', ['foo']]); | |||
# We try to find all nodes corresponding to keyword (here 't') | |||
my $nodes = $document->find(sub { | |||
my ($topnode, $element) = @_; | |||
# Filter out non-valid keywords | |||
return 0 unless ($element->isa('PPI::Token::Quote::Single')); | |||
return 0 unless (grep {$element->content eq qq{'$_'}} @keywords); | |||
# keyword (e.g. 't') should be the first element of the arrayref | |||
# passed to $stash->get() | |||
return 0 if $element->sprevious_sibling; | |||
return 0 unless $element->snext_sibling | |||
&& $element->snext_sibling->snext_sibling | |||
&& $element->snext_sibling->snext_sibling->isa('PPI::Structure::Constructor'); | |||
# Check that it's indeed a call to $stash->get() | |||
my $statement = $element->statement->parent->statement->parent->statement; | |||
return 0 unless grep { $_->isa('PPI::Token::Symbol') && $_->content eq '$stash' } $statement->children; | |||
return 0 unless grep { $_->isa('PPI::Token::Operator') && $_->content eq '->' } $statement->children; | |||
return 0 unless grep { $_->isa('PPI::Token::Word') && $_->content eq 'get' } $statement->children; | |||
return 1; | |||
}); | |||
next unless $nodes; | |||
# Write the Perl equivalent of calls to t* functions family, so | |||
# xgettext can extract the strings correctly | |||
foreach my $node (@$nodes) { | |||
my @args = map { | |||
$_->significant && !$_->isa('PPI::Token::Operator') ? $_->content : () | |||
} $node->snext_sibling->snext_sibling->find_first('PPI::Statement')->children; | |||
my $keyword = $node->content; | |||
$keyword =~ s/^'t(.*)'$/__$1/; | |||
# Only keep required args to have a clean output | |||
my @required_args = shift @args; | |||
push @required_args, shift @args if $keyword =~ /n/; | |||
push @required_args, shift @args if $keyword =~ /p/; | |||
say $fh "$keyword(" . join(', ', @required_args) . ");"; | |||
} | |||
} | |||
close $fh; | |||
} | |||
return $tempdir; | |||
} | |||
sub extract_messages { | |||
my $self = shift; | |||
say "Extract messages into POT file" if $self->{verbose}; | |||
my $intranetdir = $self->{context}->config('intranetdir'); | |||
my $opacdir = $self->{context}->config('opacdir'); | |||
# Find common ancestor directory | |||
my @intranetdirs = File::Spec->splitdir($intranetdir); | |||
my @opacdirs = File::Spec->splitdir($opacdir); | |||
my @basedirs; | |||
while (@intranetdirs and @opacdirs) { | |||
my ($dir1, $dir2) = (shift @intranetdirs, shift @opacdirs); | |||
last if $dir1 ne $dir2; | |||
push @basedirs, $dir1; | |||
} | |||
my $basedir = File::Spec->catdir(@basedirs); | |||
my @files_to_scan; | |||
my @directories_to_scan = ('.'); | |||
my @blacklist = map { File::Spec->catdir(@intranetdirs, $_) } qw(blib koha-tmpl skel tmp t); | |||
while (@directories_to_scan) { | |||
my $dir = shift @directories_to_scan; | |||
opendir DIR, File::Spec->catdir($basedir, $dir) or die "Unable to open $dir: $!"; | |||
foreach my $entry (readdir DIR) { | |||
next if $entry =~ /^\./; | |||
my $relentry = File::Spec->catfile($dir, $entry); | |||
my $abspath = File::Spec->catfile($basedir, $relentry); | |||
if (-d $abspath and not grep { $_ eq $relentry } @blacklist) { | |||
push @directories_to_scan, $relentry; | |||
} elsif (-f $abspath and $relentry =~ /\.(pl|pm)$/) { | |||
push @files_to_scan, $relentry; | |||
} | |||
} | |||
} | |||
my $intrahtdocs = $self->{context}->config('intrahtdocs'); | |||
my $opachtdocs = $self->{context}->config('opachtdocs'); | |||
my @intranet_tt_files; | |||
find(sub { | |||
if ($File::Find::dir =~ m|/en/| && $_ =~ m/\.(tt|inc)$/) { | |||
my $filename = $File::Find::name; | |||
$filename =~ s|^$intrahtdocs/||; | |||
push @intranet_tt_files, $filename; | |||
} | |||
}, $intrahtdocs); | |||
my @opac_tt_files; | |||
find(sub { | |||
if ($File::Find::dir =~ m|/en/| && $_ =~ m/\.(tt|inc)$/) { | |||
my $filename = $File::Find::name; | |||
$filename =~ s|^$opachtdocs/||; | |||
push @opac_tt_files, $filename; | |||
} | |||
}, $opachtdocs); | |||
my $tempdir = tempdir('Koha-translate-XXXX', TMPDIR => 1, CLEANUP => 1); | |||
$self->extract_messages_from_templates($tempdir, 'intranet', @intranet_tt_files); | |||
$self->extract_messages_from_templates($tempdir, 'opac', @opac_tt_files); | |||
@intranet_tt_files = map { File::Spec->catfile('koha-tmpl', 'intranet-tmpl', $_) } @intranet_tt_files; | |||
@opac_tt_files = map { File::Spec->catfile('koha-tmpl', 'opac-tmpl', $_) } @opac_tt_files; | |||
my @tt_files = grep { -e File::Spec->catfile($tempdir, $_) } @intranet_tt_files, @opac_tt_files; | |||
push @files_to_scan, @tt_files; | |||
my $xgettext_common_args = "--force-po --from-code=UTF-8 " | |||
. "--package-name=Koha --package-version='' " | |||
. "-k -k__ -k__x -k__n:1,2 -k__nx:1,2 -k__xn:1,2 -k__p:1c,2 " | |||
. "-k__px:1c,2 -k__np:1c,2,3 -k__npx:1c,2,3 -kN__ -kN__n:1,2 " | |||
. "-kN__p:1c,2 -kN__np:1c,2,3 "; | |||
my $xgettext_cmd = "$self->{xgettext} -L Perl $xgettext_common_args " | |||
. "-o $Bin/$self->{domain}.pot -D $tempdir -D $basedir"; | |||
$xgettext_cmd .= " $_" foreach (@files_to_scan); | |||
if (system($xgettext_cmd) != 0) { | |||
die "system call failed: $xgettext_cmd"; | |||
} | |||
my @js_dirs = ( | |||
"$intrahtdocs/prog/js", | |||
"$opachtdocs/bootstrap/js", | |||
); | |||
my @js_files; | |||
find(sub { | |||
if ($_ =~ m/\.js$/) { | |||
my $filename = $File::Find::name; | |||
$filename =~ s|^$intranetdir/||; | |||
push @js_files, $filename; | |||
} | |||
}, @js_dirs); | |||
$xgettext_cmd = "$self->{xgettext} -L JavaScript $xgettext_common_args " | |||
. "-o $Bin/$self->{domain}-js.pot -D $intranetdir"; | |||
$xgettext_cmd .= " $_" foreach (@js_files); | |||
if (system($xgettext_cmd) != 0) { | |||
die "system call failed: $xgettext_cmd"; | |||
} | |||
my $replace_charset_cmd = "$self->{sed} --in-place " . | |||
"--expression='s/charset=CHARSET/charset=UTF-8/' " . | |||
"$Bin/$self->{domain}.pot $Bin/$self->{domain}-js.pot"; | |||
if (system($replace_charset_cmd) != 0) { | |||
die "system call failed: $replace_charset_cmd"; | |||
} | |||
} | |||
sub install_messages { | |||
my ($self) = @_; | |||
@@ -1012,8 +435,9 @@ sub install_messages { | |||
my $js_pofile = "$self->{path_po}/$self->{lang}-messages-js.po"; | |||
unless ( -f $pofile && -f $js_pofile ) { | |||
$self->create_messages(); | |||
die "PO files for language '$self->{lang}' do not exist"; | |||
} | |||
say "Install messages ($locale)" if $self->{verbose}; | |||
make_path($modir); | |||
system "$self->{msgfmt} -o $mofile $pofile"; | |||
@@ -1035,13 +459,6 @@ sub install_messages { | |||
} | |||
} | |||
sub remove_pot { | |||
my $self = shift; | |||
unlink "$Bin/$self->{domain}.pot"; | |||
unlink "$Bin/$self->{domain}-js.pot"; | |||
} | |||
sub compress { | |||
my ($self, $files) = @_; | |||
my @langs = $self->{lang} ? ($self->{lang}) : $self->get_all_langs(); | |||
@@ -1074,11 +491,15 @@ sub install { | |||
my ($self, $files) = @_; | |||
return unless $self->{lang}; | |||
$self->uncompress(); | |||
$self->install_tmpl($files) unless $self->{pref_only}; | |||
$self->install_prefs(); | |||
$self->install_messages(); | |||
$self->remove_pot(); | |||
$self->install_installer(); | |||
if ($self->{pref_only}) { | |||
$self->install_prefs(); | |||
} else { | |||
$self->install_tmpl($files); | |||
$self->install_prefs(); | |||
$self->install_messages(); | |||
$self->install_installer(); | |||
} | |||
} | |||
@@ -1090,34 +511,6 @@ sub get_all_langs { | |||
@files = map { $_ =~ s/-pref.(po|po.gz)$//r } @files; | |||
} | |||
sub update { | |||
my ($self, $files) = @_; | |||
my @langs = $self->{lang} ? ($self->{lang}) : $self->get_all_langs(); | |||
for my $lang ( @langs ) { | |||
$self->set_lang( $lang ); | |||
$self->uncompress(); | |||
$self->update_tmpl($files) unless $self->{pref_only}; | |||
$self->update_prefs(); | |||
$self->update_messages(); | |||
$self->update_installer(); | |||
} | |||
$self->remove_pot(); | |||
} | |||
sub create { | |||
my ($self, $files) = @_; | |||
return unless $self->{lang}; | |||
$self->create_tmpl($files) unless $self->{pref_only}; | |||
$self->create_prefs(); | |||
$self->create_messages(); | |||
$self->remove_pot(); | |||
$self->create_installer(); | |||
} | |||
1; | |||
@@ -1,5 +1,13 @@ | |||
msgid "" | |||
msgstr "Project-Id-Version: PACKAGE VERSION\\nPO-Revision-Date: YEAR-MO-DA HO:MI +ZONE\\nLast-Translator: FULL NAME <EMAIL@ADDRESS>\\nLanguage-Team: Koha Translate List <koha-translate@lists.koha-community.org>\\nMIME-Version: 1.0\\nContent-Type: text/plain; charset=UTF-8\\nContent-Transfer-Encoding: 8bit\\nPlural-Forms: nplurals=2; plural=(n > 1);\\n" | |||
msgstr "" | |||
"Project-Id-Version: PACKAGE VERSION\n" | |||
"PO-Revision-Date: YEAR-MO-DA HO:MI +ZONE\n" | |||
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n" | |||
"Language-Team: Koha Translate List <koha-translate@lists.koha-community.org>\n" | |||
"MIME-Version: 1.0\n" | |||
"Content-Type: text/plain; charset=UTF-8\n" | |||
"Content-Transfer-Encoding: 8bit\n" | |||
"Plural-Forms: nplurals=2; plural=(n > 1);\n" | |||
# Accounting | |||
msgid "accounting.pref" | |||
@@ -1,5 +1,13 @@ | |||
msgid "" | |||
msgstr "Project-Id-Version: PACKAGE VERSION\\nPO-Revision-Date: YEAR-MO-DA HO:MI +ZONE\\nLast-Translator: FULL NAME <EMAIL@ADDRESS>\\nLanguage-Team: Koha Translate List <koha-translate@lists.koha-community.org>\\nMIME-Version: 1.0\\nContent-Type: text/plain; charset=UTF-8\\nContent-Transfer-Encoding: 8bit\\nPlural-Forms: nplurals=2; plural=(n > 1);\\n" | |||
msgstr "" | |||
"Project-Id-Version: PACKAGE VERSION\n" | |||
"PO-Revision-Date: YEAR-MO-DA HO:MI +ZONE\n" | |||
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n" | |||
"Language-Team: Koha Translate List <koha-translate@lists.koha-community.org>\n" | |||
"MIME-Version: 1.0\n" | |||
"Content-Type: text/plain; charset=UTF-8\n" | |||
"Content-Transfer-Encoding: 8bit\n" | |||
"Plural-Forms: nplurals=2; plural=(n > 1);\n" | |||
# Accounting | |||
msgid "accounting.pref" | |||
@@ -1,5 +1,13 @@ | |||
msgid "" | |||
msgstr "Project-Id-Version: PACKAGE VERSION\\nPO-Revision-Date: YEAR-MO-DA HO:MI +ZONE\\nLast-Translator: FULL NAME <EMAIL@ADDRESS>\\nLanguage-Team: Koha Translate List <koha-translate@lists.koha-community.org>\\nMIME-Version: 1.0\\nContent-Type: text/plain; charset=UTF-8\\nContent-Transfer-Encoding: 8bit\\nPlural-Forms: nplurals=2; plural=(n > 1);\\n" | |||
msgstr "" | |||
"Project-Id-Version: PACKAGE VERSION\n" | |||
"PO-Revision-Date: YEAR-MO-DA HO:MI +ZONE\n" | |||
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n" | |||
"Language-Team: Koha Translate List <koha-translate@lists.koha-community.org>\n" | |||
"MIME-Version: 1.0\n" | |||
"Content-Type: text/plain; charset=UTF-8\n" | |||
"Content-Transfer-Encoding: 8bit\n" | |||
"Plural-Forms: nplurals=2; plural=(n > 1);\n" | |||
# Accounting | |||
msgid "accounting.pref" | |||
@@ -1,5 +1,13 @@ | |||
msgid "" | |||
msgstr "Project-Id-Version: PACKAGE VERSION\\nPO-Revision-Date: YEAR-MO-DA HO:MI +ZONE\\nLast-Translator: FULL NAME <EMAIL@ADDRESS>\\nLanguage-Team: Koha Translate List <koha-translate@lists.koha-community.org>\\nMIME-Version: 1.0\\nContent-Type: text/plain; charset=UTF-8\\nContent-Transfer-Encoding: 8bit\\nPlural-Forms: nplurals=2; plural=(n > 1);\\n" | |||
msgstr "" | |||
"Project-Id-Version: PACKAGE VERSION\n" | |||
"PO-Revision-Date: YEAR-MO-DA HO:MI +ZONE\n" | |||
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n" | |||
"Language-Team: Koha Translate List <koha-translate@lists.koha-community.org>\n" | |||
"MIME-Version: 1.0\n" | |||
"Content-Type: text/plain; charset=UTF-8\n" | |||
"Content-Transfer-Encoding: 8bit\n" | |||
"Plural-Forms: nplurals=2; plural=(n > 1);\n" | |||
# Accounting | |||
msgid "accounting.pref" | |||
@@ -204,11 +204,9 @@ sub usage { | |||
my($exitcode) = @_; | |||
my $h = $exitcode? *STDERR: *STDOUT; | |||
print $h <<EOF; | |||
Usage: $0 create [OPTION] | |||
or: $0 update [OPTION] | |||
or: $0 install [OPTION] | |||
Usage: $0 install [OPTION] | |||
or: $0 --help | |||
Create or update PO files from templates, or install translated templates. | |||
Install translated templates. | |||
-i, --input=SOURCE Get or update strings from SOURCE directory(s). | |||
On create or update can have multiple values. | |||
@@ -230,7 +228,6 @@ Create or update PO files from templates, or install translated templates. | |||
--help Display this help and exit | |||
-q, --quiet no output to screen (except for errors) | |||
The -o option is ignored for the "create" and "update" actions. | |||
Try `perldoc $0` for perhaps more information. | |||
EOF | |||
exit($exitcode); | |||
@@ -265,12 +262,6 @@ GetOptions( | |||
VerboseWarnings::set_application_name($0); | |||
VerboseWarnings::set_pedantic_mode($pedantic_p); | |||
# keep the buggy Locale::PO quiet if it says stupid things | |||
$SIG{__WARN__} = sub { | |||
my($s) = @_; | |||
print STDERR $s unless $s =~ /^Strange line in [^:]+: #~/s | |||
}; | |||
my $action = shift or usage_error('You must specify an ACTION.'); | |||
usage_error('You must at least specify input and string list filenames.') | |||
if !@in_dirs || !defined $str_file; | |||
@@ -344,89 +335,9 @@ if (!defined $charset_out) { | |||
$charset_out = TmplTokenizer::charset_canon('UTF-8'); | |||
warn "Warning: Charset Out defaulting to $charset_out\n" unless ( $quiet ); | |||
} | |||
my $xgettext = './xgettext.pl'; # actual text extractor script | |||
my $st; | |||
if ($action eq 'create') { | |||
# updates the list. As the list is empty, every entry will be added | |||
if (!-s $str_file) { | |||
warn "Removing empty file $str_file\n" unless ( $quiet ); | |||
unlink $str_file || die "$str_file: $!\n"; | |||
} | |||
die "$str_file: Output file already exists\n" if -f $str_file; | |||
my($tmph1, $tmpfile1) = tmpnam(); | |||
my($tmph2, $tmpfile2) = tmpnam(); | |||
close $tmph2; # We just want a name | |||
# Generate the temporary file that acts as <MODULE>/POTFILES.in | |||
for my $input (@in_files) { | |||
print $tmph1 "$input\n"; | |||
} | |||
close $tmph1; | |||
warn "I $charset_in O $charset_out" unless ( $quiet ); | |||
# Generate the specified po file ($str_file) | |||
$st = system ($xgettext, '-s', '-f', $tmpfile1, '-o', $tmpfile2, | |||
(defined $charset_in? ('-I', $charset_in): ()), | |||
(defined $charset_out? ('-O', $charset_out): ()) | |||
); | |||
# Run msgmerge so that the pot file looks like a real pot file | |||
# We need to help msgmerge a bit by pre-creating a dummy po file that has | |||
# the headers and the "" msgid & msgstr. It will fill in the rest. | |||
if ($st == 0) { | |||
# Merge the temporary "pot file" with the specified po file ($str_file) | |||
# FIXME: msgmerge(1) is a Unix dependency | |||
# FIXME: need to check the return value | |||
unless (-f $str_file) { | |||
open(my $infh, '<', $tmpfile2); | |||
open(my $outfh, '>', $str_file); | |||
while (<$infh>) { | |||
print $outfh $_; | |||
last if /^\n/s; | |||
} | |||
close $infh; | |||
close $outfh; | |||
} | |||
$st = system("msgmerge ".($quiet?'-q':'')." -s $str_file $tmpfile2 -o - | msgattrib --no-obsolete -o $str_file"); | |||
} else { | |||
error_normal("Text extraction failed: $xgettext: $!\n", undef); | |||
error_additional("Will not run msgmerge\n", undef); | |||
} | |||
unlink $tmpfile1 || warn_normal("$tmpfile1: unlink failed: $!\n", undef); | |||
unlink $tmpfile2 || warn_normal("$tmpfile2: unlink failed: $!\n", undef); | |||
} elsif ($action eq 'update') { | |||
my($tmph1, $tmpfile1) = tmpnam(); | |||
my($tmph2, $tmpfile2) = tmpnam(); | |||
close $tmph2; # We just want a name | |||
# Generate the temporary file that acts as <MODULE>/POTFILES.in | |||
for my $input (@in_files) { | |||
print $tmph1 "$input\n"; | |||
} | |||
close $tmph1; | |||
# Generate the temporary file that acts as <MODULE>/<LANG>.pot | |||
$st = system($xgettext, '-s', '-f', $tmpfile1, '-o', $tmpfile2, | |||
'--po-mode', | |||
(defined $charset_in? ('-I', $charset_in): ()), | |||
(defined $charset_out? ('-O', $charset_out): ())); | |||
if ($st == 0) { | |||
# Merge the temporary "pot file" with the specified po file ($str_file) | |||
# FIXME: msgmerge(1) is a Unix dependency | |||
# FIXME: need to check the return value | |||
if ( @filenames ) { | |||
my ($tmph3, $tmpfile3) = tmpnam(); | |||
$st = system("msgcat $str_file $tmpfile2 > $tmpfile3"); | |||
$st = system("msgmerge ".($quiet?'-q':'')." -s $str_file $tmpfile3 -o - | msgattrib --no-obsolete -o $str_file") | |||
unless $st; | |||
} else { | |||
$st = system("msgmerge ".($quiet?'-q':'')." -s $str_file $tmpfile2 -o - | msgattrib --no-obsolete -o $str_file"); | |||
} | |||
} else { | |||
error_normal("Text extraction failed: $xgettext: $!\n", undef); | |||
error_additional("Will not run msgmerge\n", undef); | |||
} | |||
unlink $tmpfile1 || warn_normal("$tmpfile1: unlink failed: $!\n", undef); | |||
unlink $tmpfile2 || warn_normal("$tmpfile2: unlink failed: $!\n", undef); | |||
} elsif ($action eq 'install') { | |||
if ($action eq 'install') { | |||
if(!defined($out_dir)) { | |||
usage_error("You must specify an output directory when using the install method."); | |||
} | |||
@@ -554,14 +465,6 @@ translation, it can be suppressed with the %0.0s notation. | |||
Using the PO format also means translators can add their | |||
own comments in the translation files, if necessary. | |||
=item - | |||
Create, update, and install actions are all based on the | |||
same scanner module. This ensures that update and install | |||
have the same idea of what is a translatable string; | |||
attribute names in tags, for example, will not be | |||
accidentally translated. | |||
=back | |||
=head1 NOTES | |||
@@ -569,22 +472,8 @@ accidentally translated. | |||
Anchors are represented by an <AI<n>> notation. | |||
The meaning of this non-standard notation might not be obvious. | |||
The create action calls xgettext.pl to do the actual work; | |||
the update action calls xgettext.pl, msgmerge(1) and msgattrib(1) | |||
to do the actual work. | |||
=head1 BUGS | |||
xgettext.pl must be present in the current directory; both | |||
msgmerge(1) and msgattrib(1) must also be present in the search path. | |||
The script currently does not check carefully whether these | |||
dependent commands are present. | |||
Locale::PO(3) has a lot of bugs. It can neither parse nor | |||
generate GNU PO files properly; a couple of workarounds have | |||
been written in TmplTokenizer and more is likely to be needed | |||
(e.g., to get rid of the "Strange line" warning for #~). | |||
This script may not work in Windows. | |||
There are probably some other bugs too, since this has not been | |||
@@ -592,12 +481,7 @@ tested very much. | |||
=head1 SEE ALSO | |||
xgettext.pl, | |||
TmplTokenizer.pm, | |||
msgmerge(1), | |||
Locale::PO(3), | |||
translator_doc.txt | |||
http://www.saas.nsw.edu.au/koha_wiki/index.php?page=DifficultTerms | |||
=cut |
@@ -54,14 +54,13 @@ usage() if $#ARGV != 1 && $#ARGV != 0; | |||
my ($cmd, $lang) = @ARGV; | |||
$cmd = lc $cmd; | |||
if ( $cmd =~ /^(create|install|update|compress|uncompress)$/ ) { | |||
if ( $cmd =~ /^(install|compress|uncompress)$/ ) { | |||
my $installer = LangInstaller->new( $lang, $pref, $verbose ); | |||
if ( $cmd ne 'create' and $lang and not grep( {$_ eq $lang} @{ $installer->{langs} } ) ) { | |||
if ( $lang and not grep( {$_ eq $lang} @{ $installer->{langs} } ) ) { | |||
print "Unsupported language: $lang\n"; | |||
exit; | |||
} | |||
if ( $all ) { | |||
usage() if $cmd eq 'create'; | |||
for my $lang ( @{$installer->{langs}} ) { | |||
$installer->set_lang( $lang ); | |||
$installer->$cmd(\@files); | |||
@@ -71,9 +70,19 @@ if ( $cmd =~ /^(create|install|update|compress|uncompress)$/ ) { | |||
$installer->$cmd(\@files); | |||
} | |||
Koha::Caches->get_instance()->flush_all if $cmd ne 'update'; | |||
} | |||
else { | |||
Koha::Caches->get_instance()->flush_all; | |||
} elsif ($cmd eq 'create' or $cmd eq 'update') { | |||
my $command = "gulp po:$cmd"; | |||
$command .= " --silent" if (!$verbose); | |||
$command .= " --lang $lang" if ($lang); | |||
if ($verbose) { | |||
print STDERR "Deprecation notice: PO creation and update are now gulp tasks. See docs/development/internationalization.md\n"; | |||
print STDERR "Running `$command`\n"; | |||
} | |||
system($command); | |||
} else { | |||
usage(); | |||
} | |||
@@ -85,12 +94,9 @@ translate - Handle templates and preferences translation | |||
=head1 SYNOPSYS | |||
translate create fr-FR | |||
translate update fr-FR | |||
translate install fr-FR | |||
translate install fr-FR -f search -f memberentry | |||
translate -p install fr-FR | |||
translate install | |||
translate compress [fr-FR] | |||
translate uncompress [fr-FR] | |||
@@ -98,7 +104,7 @@ translate - Handle templates and preferences translation | |||
In Koha, three categories of information are translated based on standard GNU | |||
.po files: opac templates pages, intranet templates and system preferences. The | |||
script is a wrapper. It allows to quickly create/update/install .po files for a | |||
script is a wrapper. It allows to quickly install .po files for a | |||
given language or for all available languages. | |||
=head1 USAGE | |||
@@ -107,38 +113,6 @@ Use the -v or --verbose parameter to make translator more verbose. | |||
=over | |||
=item translate create F<lang> | |||
Create 3 .po files in F</misc/translator/po> subdirectory: (1) from opac pages | |||
templates, (2) intranet templates, and (3) from preferences. English 'en' | |||
version of templates and preferences are used as references. | |||
=over | |||
=item F<lang>-opac-{theme}.po | |||
Contains extracted text from english (en) OPAC templates found in | |||
<KOHA_ROOT>/koha-tmpl/opac-tmpl/{theme}/en/ directory. | |||
=item F<lang>-intranet.po | |||
Contains extracted text from english (en) intranet templates found in | |||
<KOHA_ROOT>/koha-tmpl/intranet-tmpl/prog/en/ directory. | |||
=item F<lang>-pref.po | |||
Contains extracted text from english (en) preferences. They are found in files | |||
located in <KOHA_ROOT>/koha-tmpl/intranet-tmpl/prog/en/admin/preferences | |||
directory. | |||
=back | |||
=item translate [-p] update F<lang> | |||
Update .po files in F<po> directory, named F<lang>-*.po. Without F<lang>, all | |||
available languages are updated. With -p option, only preferences .po file is | |||
updated. | |||
=item translate [-p|-f] install F<lang> | |||
Use .po files to translate the english version of templates and preferences files | |||
@@ -0,0 +1,158 @@ | |||
#!/usr/bin/perl | |||
# This file is part of Koha. | |||
# | |||
# Koha is free software; you can redistribute it and/or modify it | |||
# under the terms of the GNU General Public License as published by | |||
# the Free Software Foundation; either version 3 of the License, or | |||
# (at your option) any later version. | |||
# | |||
# Koha is distributed in the hope that it will be useful, but | |||
# WITHOUT ANY WARRANTY; without even the implied warranty of | |||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | |||
# GNU General Public License for more details. | |||
# | |||
# You should have received a copy of the GNU General Public License | |||
# along with Koha; if not, see <http://www.gnu.org/licenses>. | |||
=head1 NAME | |||
xgettext-installer - extract translatable strings from installer YAML files | |||
=head1 SYNOPSIS | |||
xgettext-installer [OPTION] [INPUTFILE]... | |||
=head1 OPTIONS | |||
=over | |||
=item B<-f, --files-from=FILE> | |||
get list of input files from FILE | |||
=item B<-o, --output=FILE> | |||
write output to the specified file | |||
=item B<-h, --help> | |||
display this help and exit | |||
=back | |||
=cut | |||
use Modern::Perl; | |||
use Getopt::Long; | |||
use Locale::PO; | |||
use Pod::Usage; | |||
use YAML::Syck qw(LoadFile); | |||
$YAML::Syck::ImplicitTyping = 1; | |||
my $output = 'messages.pot'; | |||
my $files_from; | |||
my $help; | |||
GetOptions( | |||
'output=s' => \$output, | |||
'files-from=s' => \$files_from, | |||
'help' => \$help, | |||
) or pod2usage(-verbose => 1, -exitval => 2); | |||
if ($help) { | |||
pod2usage(-verbose => 1, -exitval => 0); | |||
} | |||
my @files = @ARGV; | |||
if ($files_from) { | |||
open(my $fh, '<', $files_from) or die "Cannot open $files_from: $!"; | |||
push @files, <$fh>; | |||
chomp @files; | |||
close $fh; | |||
} | |||
my $pot = { | |||
'' => Locale::PO->new( | |||
-msgid => '', | |||
-msgstr => | |||
"Project-Id-Version: Koha\n" | |||
. "PO-Revision-Date: YEAR-MO-DA HO:MI +ZONE\n" | |||
. "Last-Translator: FULL NAME <EMAIL\@ADDRESS>\n" | |||
. "Language-Team: Koha Translate List <koha-translate\@lists.koha-community.org>\n" | |||
. "MIME-Version: 1.0\n" | |||
. "Content-Type: text/plain; charset=UTF-8\n" | |||
. "Content-Transfer-Encoding: 8bit\n" | |||
), | |||
}; | |||
for my $file (@files) { | |||
my $yaml = LoadFile($file); | |||
my @tables = @{ $yaml->{'tables'} }; | |||
my $tablec = 0; | |||
for my $table (@tables) { | |||
$tablec++; | |||
my $table_name = ( keys %$table )[0]; | |||
my @translatable = @{ $table->{$table_name}->{translatable} }; | |||
my @rows = @{ $table->{$table_name}->{rows} }; | |||
my @multiline = @{ $table->{$table_name}->{'multiline'} }; | |||
my $rowc = 0; | |||
for my $row (@rows) { | |||
$rowc++; | |||
for my $field (@translatable) { | |||
if ( @multiline and grep { $_ eq $field } @multiline ) { | |||
# multiline fields, only notices ATM | |||
my $mulc; | |||
foreach my $line ( @{ $row->{$field} } ) { | |||
$mulc++; | |||
# discard pure html, TT, empty | |||
next if ( $line =~ /^(\s*<.*?>\s*$|^\s*\[.*?\]\s*|\s*)$/ ); | |||
# put placeholders | |||
$line =~ s/(<<.*?>>|\[\%.*?\%\]|<.*?>)/\%s/g; | |||
# discard non strings | |||
next if ( $line =~ /^(\s|%s|-|[[:punct:]]|\(|\))*$/ or length($line) < 2 ); | |||
if ( not $pot->{$line} ) { | |||
my $msg = new Locale::PO( | |||
-msgid => $line, | |||
-msgstr => '', | |||
-reference => "$file:$table_name:$tablec:row:$rowc:mul:$mulc" | |||
); | |||
$pot->{$line} = $msg; | |||
} | |||
} | |||
} elsif (defined $row->{$field} && length($row->{$field}) > 1 && !$pot->{ $row->{$field} }) { | |||
my $msg = new Locale::PO( | |||
-msgid => $row->{$field}, | |||
-msgstr => '', | |||
-reference => "$file:$table_name:$tablec:row:$rowc" | |||
); | |||
$pot->{ $row->{$field} } = $msg; | |||
} | |||
} | |||
} | |||
} | |||
my $desccount = 0; | |||
for my $description ( @{ $yaml->{'description'} } ) { | |||
$desccount++; | |||
if ( length($description) > 1 and not $pot->{$description} ) { | |||
my $msg = new Locale::PO( | |||
-msgid => $description, | |||
-msgstr => '', | |||
-reference => "$file:description:$desccount" | |||
); | |||
$pot->{$description} = $msg; | |||
} | |||
} | |||
} | |||
Locale::PO->save_file_fromhash($output, $pot); |
@@ -0,0 +1,151 @@ | |||
#!/usr/bin/perl | |||
# This file is part of Koha. | |||
# | |||
# Koha is free software; you can redistribute it and/or modify it | |||
# under the terms of the GNU General Public License as published by | |||
# the Free Software Foundation; either version 3 of the License, or | |||
# (at your option) any later version. | |||
# | |||
# Koha is distributed in the hope that it will be useful, but | |||
# WITHOUT ANY WARRANTY; without even the implied warranty of | |||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | |||
# GNU General Public License for more details. | |||
# | |||
# You should have received a copy of the GNU General Public License | |||
# along with Koha; if not, see <http://www.gnu.org/licenses>. | |||
=head1 NAME | |||
xgettext-pref - extract translatable strings from system preferences YAML files | |||
=head1 SYNOPSIS | |||
xgettext-pref [OPTION] [INPUTFILE]... | |||
=head1 OPTIONS | |||
=over | |||
=item B<-f, --files-from=FILE> | |||
get list of input files from FILE | |||
=item B<-o, --output=FILE> | |||
write output to the specified file | |||
=item B<-h, --help> | |||
display this help and exit | |||
=back | |||
=cut | |||
use Modern::Perl; | |||
use File::Basename; | |||
use Getopt::Long; | |||
use Locale::PO; | |||
use Pod::Usage; | |||
use YAML::Syck qw(LoadFile); | |||
$YAML::Syck::ImplicitTyping = 1; | |||
my $output = 'messages.pot'; | |||
my $files_from; | |||
my $help; | |||
GetOptions( | |||
'output=s' => \$output, | |||
'files-from=s' => \$files_from, | |||
'help' => \$help, | |||
) or pod2usage(-verbose => 1, -exitval => 2); | |||
if ($help) { | |||
pod2usage(-verbose => 1, -exitval => 0); | |||
} | |||
my @files = @ARGV; | |||
if ($files_from) { | |||
open(my $fh, '<', $files_from) or die "Cannot open $files_from: $!"; | |||
push @files, <$fh>; | |||
chomp @files; | |||
close $fh; | |||
} | |||
my $pot = { | |||
'' => Locale::PO->new( | |||
-msgid => '', | |||
-msgstr => "Project-Id-Version: Koha\n" | |||
. "PO-Revision-Date: YEAR-MO-DA HO:MI +ZONE\n" | |||
. "Last-Translator: FULL NAME <EMAIL\@ADDRESS>\n" | |||
. "Language-Team: Koha Translate List <koha-translate\@lists.koha-community.org>\n" | |||
. "MIME-Version: 1.0\n" | |||
. "Content-Type: text/plain; charset=UTF-8\n" | |||
. "Content-Transfer-Encoding: 8bit\n" | |||
), | |||
}; | |||
for my $file (@files) { | |||
my $pref = LoadFile($file); | |||
while ( my ($tab, $tab_content) = each %$pref ) { | |||
add_po(undef, basename($file)); | |||
if ( ref($tab_content) eq 'ARRAY' ) { | |||
add_prefs( $file, $tab, $tab_content ); | |||
} else { | |||
while ( my ($section, $sysprefs) = each %$tab_content ) { | |||
my $context = "$tab > $section"; | |||
my $msgid = sprintf('%s %s', basename($file), $section); | |||
add_po($tab, $msgid); | |||
add_prefs( $file, $context, $sysprefs ); | |||
} | |||
} | |||
} | |||
} | |||
Locale::PO->save_file_fromhash($output, $pot); | |||
sub add_prefs { | |||
my ($file, $context, $prefs) = @_; | |||
for my $pref (@$prefs) { | |||
my $pref_name = ''; | |||
for my $element (@$pref) { | |||
if ( ref($element) eq 'HASH' ) { | |||
$pref_name = $element->{pref}; | |||
last; | |||
} | |||
} | |||
for my $element (@$pref) { | |||
if ( ref($element) eq 'HASH' ) { | |||
while ( my ( $key, $value ) = each(%$element) ) { | |||
next unless $key eq 'choices' or $key eq 'multiple'; | |||
next unless ref($value) eq 'HASH'; | |||
for my $ckey ( keys %$value ) { | |||
my $msgid = sprintf('%s#%s# %s', basename($file), $pref_name, $value->{$ckey}); | |||
add_po( "$context > $pref_name", $msgid ); | |||
} | |||
} | |||
} | |||
elsif ($element) { | |||
my $msgid = sprintf('%s#%s# %s', basename($file), $pref_name, $element); | |||
add_po( "$context > $pref_name", $msgid ); | |||
} | |||
} | |||
} | |||
} | |||
sub add_po { | |||
my ($comment, $msgid ) = @_; | |||
return unless $msgid; | |||
$pot->{$msgid} = Locale::PO->new( | |||
-comment => $comment, | |||
-msgid => $msgid, | |||
-msgstr => '' |