This patch removes a regex that discard lines in multiline YAML values
On close inspection, there is no need for it.
To test:
1) go to misc translator, update some language
./translate update fr-CA
2) check missing string
egrep "You may pick up your article" po/fr-CA-installer.po
from sample_notices.yaml
3) apply the patch, repeat 1)
4) repeat 2), verify the string is present in the translation file
5) translate the new string, install the language,
verify string is present in the translated file
./translate install fr-CA
check fr-CA/mandatory/sample_notices.yml
There are three new strings
msgid "%sDear %s %s,%s"
msgid "%s%s%sTitle: %s"
msgid "%sYou may pick up your article at %s.%sYou can download the scanned materials via the following url(s): %s.%s"
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
There is a badly crafted regex used when extracting
strings in multiline fields in yaml files
The regex is my own, introduced in Bug 24262, sorry.
This patch correct it a little. Better eyes are welcome.
To test:
1) go to misc translator, update some language
./translate update fr-CA
2) check missing strings
egrep "Total out|Operator ID|August 31" po/fr-CA-installer.po
first two are from sample notices, third from sample creator data
3) apply the patch, repeat 1)
4) repeat 2), verify the strings are present in the translation file
5) translate some of the new strings, install the language,
verify strings are present in the translated files
./translate install fr-CA
check fr-CA/optional/sample_creator_data.yml and
fr-CA/mandatory/sample_notices.yml
There are some 60+ new strings.
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch adds a new translation file, *-installer-UNIMARC.po
To test:
1. On top of all previous patches
2. Apply this patch
3. Create (or update) some language
Ej.
misc/translation/translate create xx-YY
verify new file misc/translation/po/xx-YY-installer-UNIMARC.po
4. Install new language
misc/translation/translate install xx-YY
verify new dirs:
installer/data/mysql/xx-YY/marcflavour/unimarc/{mandatory,optional}
5. Repeat install procedure selecting xx-YY language and UNIMARC
verify all frameworks are present
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
On bug 17591 we discovered that there was something weird going on with
the way we export and use subroutines/modules.
This patch tries to standardize our EXPORT to use EXPORT_OK only.
That way we will need to explicitely define the subroutine we want to
use from a module.
This patch is a squashed version of:
Bug 17600: After export.pl
Bug 17600: After perlimport
Bug 17600: Manual changes
Bug 17600: Other manual changes after second perlimports run
Bug 17600: Fix tests
And a lot of other manual changes.
export.pl is a dirty script that can be found on bug 17600.
"perlimport" is:
git clone https://github.com/oalders/App-perlimports.git
cd App-perlimports/
cpanm --installdeps .
export PERL5LIB="$PERL5LIB:/kohadevbox/koha/App-perlimports/lib"
find . \( -name "*.pl" -o -name "*.pm" \) -exec perl App-perlimports/script/perlimports --inplace-edit --no-preserve-unused --filename {} \;
The ideas of this patch are to:
* use EXPORT_OK instead of EXPORT
* perltidy the EXPORT_OK list
* remove '&' before the subroutine names
* remove some uneeded use statements
* explicitely import the subroutines we need within the controllers or
modules
Note that the private subroutines (starting with _) should not be
exported (and not used from outside of the module except from tests).
EXPORT vs EXPORT_OK (from
https://www.thegeekstuff.com/2010/06/perl-exporter-examples/)
"""
Export allows to export the functions and variables of modules to user’s namespace using the standard import method. This way, we don’t need to create the objects for the modules to access it’s members.
@EXPORT and @EXPORT_OK are the two main variables used during export operation.
@EXPORT contains list of symbols (subroutines and variables) of the module to be exported into the caller namespace.
@EXPORT_OK does export of symbols on demand basis.
"""
If this patch caused a conflict with a patch you wrote prior to its
push:
* Make sure you are not reintroducing a "use" statement that has been
removed
* "$subroutine" is not exported by the C4::$MODULE module
means that you need to add the subroutine to the @EXPORT_OK list
* Bareword "$subroutine" not allowed while "strict subs"
means that you didn't imported the subroutine from the module:
- use $MODULE qw( $subroutine list );
You can also use the fully qualified namespace: C4::$MODULE::$subroutine
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
memcached address and namespace are in $KOHA_CONF, so it is required to
read it before being able to access the cache. And after that,
configuration is kept in memory forever. Storing this in memcached is
useless and even counter-productive, since Koha reads both the file and
the cache
This patch addresses this issue by removing the cache-related code from
C4::Context->new.
It means that C4::Context->new will always read the configuration file,
so this patch also replaces inappropriate calls to
C4::Context->new->config by appropriate calls to C4::Context->config
It also fixes a bug where C4::Context->new would ignore the filepath
given in parameters if there was something in cache.
It also removes a problematic call to Koha::Caches->get_instance.
Because this call was outside of any subroutine, it would have happened
before the initialization of $C4::Context::context (which happen in
C4::Context::import)
Test plan:
1. Do not apply the patch yet
2. Add the following line at the beginning of Koha::Config::read_from_file
warn "read_from_file($file)";
This will allow you to check how many times the file is read.
3. Flush memcached and restart starman
4. Check the logs, you should see "read_from_file" a bunch of times
5. Apply the patch
6. Re-add the line from step 2
7. Flush memcached and restart starman
8. Check the logs, you should see "read_from_file" only once
9. Make sure the memcached config from $KOHA_CONF (memcached_servers,
memcached_namespace) is taken into account by checking the About page
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
We had two problems in the LangInstaller.pm module:
1) the PO file was read as bytes instead of as a UTF-8 text stream
2) The YAML file being outputted was double encoded, once by setting
the file handle to output UTF-8 and other time in the DumpFile
function internally
To test:
1. Before applying patch do the following
$ cd misc/translator
$ ./translate update pl-PL
$ ./translate install pl-PL
$ cd -
$ less installer/data/mysql/pl-PL/marcflavour/marc21/mandatory/authorities_normal_marc21.yml
2. Notice the output of the authorities_normal_marc21.yml contains
invalid looking UTF-8 characters
3. $ git clean -d -f # to remove the old translation files
4. Apply patch and repeat the steps and notice the
authorities_normal_marc21.yml contains valid looking UTF-8 characters
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
misc/translator/translate was doing three different things:
- extract translatable strings
- create or update PO files
- install translated templates
This patch separates responsibilities by moving the string extraction
code into several 'xgettext-like' scripts and adds gulp tasks to
automate string extraction and PO files update
This has several benefits:
- gulp runs tasks in parallel, so it's a lot faster (updating all PO
files is at least 10 times faster with my 4-cores CPU)
- there is no need for $KOHA_CONF to be defined
LangInstaller.pm relied on $KOHA_CONF to get the different paths
needed. I'm not sure why, since string extraction and PO update should
work on source files, not installed files
- string extraction code can be more easily tested
This patch also brings a couple of fixes and improvements:
- TT string extraction (strings wrapped in [% t(...) %]) was done with
Template::Parser and PPI, which was extremely slow, and had some
problems (see bug 24797).
This is now done with Locale::XGettext::TT2 (new dependency) which is
a lot faster, and fixes bug 24797
- Fix header in 4 PO files
For backward compatibility, 'create' and 'update' commands of
misc/translator/translate can still be used and will execute the
corresponding gulp task
Test plan:
1. Run `yarn install` and install Locale::XGettext::TT2
2. Run `gulp po:update`
3. Verify the contents of updated PO files
4. Run `cd misc/translator && ./translate install <lang>`
5. Verify that all (templates, sysprefs, xslt, installer files) is
correctly translated
6. Run `gulp po:create --lang <lang>` and verify that it created all PO
files for that language
7. Run `prove t/misc/translator`
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Need to install yarn & gulp, no errors
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch adds a .perlcriticrc (copied from qa-test-tools) and fixes
almost all perlcrictic violations according to this .perlcriticrc
The remaining violations are silenced out by appending a '## no critic'
to the offending lines. They can still be seen by using the --force
option of perlcritic
This patch also modify t/00-testcritic.t to check all Perl files using
the new .perlcriticrc.
I'm not sure if this test script is still useful as it is now equivalent
to `perlcritic --quiet .` and it looks like it is much slower
(approximatively 5 times slower on my machine)
Test plan:
1. Run `perlcritic --quiet .` from the root directory. It should output
nothing
2. Run `perlcritic --quiet --force .`. It should output 7 errors (6
StringyEval, 1 BarewordFileHandles)
3. Run `TEST_QA=1 prove t/00-testcritic.t`
4. Read the patch. Check that all changes make sense and do not
introduce undesired behaviour
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
To see it enable verbose mode
To test:
1) Install lang with install dir
(cd misc/translator; ./tranlste install de-DE)
check message: "de-DE installer dir /...mysql/de-DE already exists."
2) Apply patch
3) Repeat 1, now message is gone
4) Repeat 1 in verbose mode (./translate install de-DE -v)
check the message now appears at the bottom.
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Alex Arnaud <alex.arnaud@biblibre.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch adds handling of null values on translatable
labels.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch adds the ability to:
1) Create new translation files from yaml installer files
2) Create installer directory for a given language
It will not create a installer directory if it already exists.
New (possible) translation files:
xx-YY-installer.po
xx-YY-installer-MARC21.po
xx-YY-installer-UNIMARC.po
Needs Bug 13897 (for yaml files)
NOTE: updated version adding ability to process multiline
fields, discard small ( < 2) strings, and discard strings
with pure html, TT or punctuation.
To test:
1) Apply patches from Bug 13897
2) Apply this patch
3) Go to misc/translation
4) Create translation files for a NEW language
$ ./translate create xx-YY
check new file 'xx-YY-installer.po'
5) Copy ../../installer/data/mysql/en/optional/auth_val.yml
into ../../installer/data/mysql/en/marcflavour/marc21/mandatory/
and ../../installer/data/mysql/en/marcflavour/unimarc/mandatory/
remove po/xx-YY*, then repeat creation
check new files 'xx-YY-installer.po', 'xx-YY-installer-MARC21.po'
and 'xx-YY-installer-UNIMARC.po'
remove all new files
6) Create for xx-YY again and try update
$ ./translate create xx-YY
edit ../../installer/data/mysql/en/optional/auth_val.yml
and change one char in one of the translatable values,
also edit 'xx-YY-installer.po', translate the same
string (in msgstr).
Do an update
$ ./translate update xx-YY
check in 'xx-YY-installer.po' a fuzzy value for the
translated value and the preservation of the translation
Fix the translation, or add a new one.
7) Create an install dir for xx-YY
./translate install xx-YY
Check new dir '../../installer/data/mysql/xx-YY/'
Check files on it
$ tree ../../installer/data/mysql/xx-YY/
and compare with ../../installer/data/mysql/en/
All installation files must be present
8) Try a new Koha install using this language in the
usual way.
Check in authorised_values table for the translated string.
9) Try create an install dir for an existing language
(eg. es-ES, de-DE or fr-FR ), eg.
./translate install de-DE
Install dir is detected and not changed, a note is printed.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Use intrahtdocs and opachtdocs in order to get the correct paths for all
kind of installations
Test plan:
1. With a dev install, run ./translator create/update/install xx-XX and
verify that there is no error message
2. With a standard install, run ./translator create/update/install xx-XX
and verify that there is no error message
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Tested for dev/single/standard installs, no errors,
just a (normal) warn from po2json.
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch adds the ability to compress/uncompress
translations files.
On update/install the files are uncompressed first
The only gain is to use less space.
To test:
1) Apply the patch
2) Go to misc/translator
3) Try it
$ ./translate compress fr-FR (check po/*.gz)
$ ./translate uncompress fr-FR ( check normal files)
4) Try again with verbose mode
$ ./translate compress fr-FR -v (list compressed files)
$ ./translate uncompress fr-FR -v (list uncompressed files)
5) Try update compressed files
$ ./translate compress fr-FR
$ ./translate update fr-FR (result is uncompressed)
6) Try install compressed files
$ ./translate compress fr-FR
$ ./translate install fr-FR
7) Try compress all
$ ./translate compress (add '-v' for verbose output)
8) Try uncompress all
$ ./translate uncompress (add '-v' for verbose output)
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
We certainly faced 3 similar bugs due to this syntax: bug 23006, bug
22941 and bug 17526.
To prevent other issues related to this syntax this patch suggests to
replace them all in one go.
Test plan:
Confirm that the 2 syntaxes are similar
Eyeball the patch and confirm that there is no typo!
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
It adds Javascript equivalent of Koha::I18N's exported subroutines, and
they are used the same way.
String extraction is done only on *.js files and require gettext 0.19
(available in Debian jessie, and also in wheezy-backports)
It adds Javascript library Gettext.js for handling translation and a
Perl script po2json to transform PO file into JSON.
Gettext.js and po2json both come from Locale::Simple.
There are several tools named po2json. It's simpler to integrate this
one into Koha than to check if the good one is installed on the system.
Locale::Simple is not needed.
To avoid polluting the global namespace too much, this patch also
introduce a global JS object named Koha and add some stuff in Koha.i18n
Test plan:
1. Add a translatable string in a JS file. For example, add this:
alert(__nx("There is one item", "There are {count} items", 3,
{count: 3}));
to staff-global.js
2. cd misc/translator && ./translate update fr-FR
3. Open misc/translator/po/fr-FR-messages-js.po, verify that your
string is present, and translate it
4. cd misc/translator && ./translate install fr-FR
5. (Optional) Verify that
koha-tmpl/intranet-tmpl/prog/fr-FR/js/locale_data.js exists and
contains your translation
6. Open your browser on the staff main page, change language and verify
that the message is translated
7. Repeat 1-6 on OPAC side
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Works well, translation is OK and test message is displayed correctly.
Current qa-tool error is a false positive.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch removes some warnings when creating new languages,
those related with uft8 and two not related.
Tipical messages are:
>Warning: Can't determine original templates' charset, defaulting to UTF-8
>Warning: Charset Out defaulting to UTF-8
>...
>Removing empty file /../xx-YY-marc-NORMARC.po
>I UTF-8 O UTF-8 at /../kohaclone/misc/translator/tmpl_process3.pl line 365.
>Use of uninitialized value $pref_name in concatenation (.) or string at /.../misc/translator/LangInstaller.pm line 197.
>Created /../misc/translator/po/xx-YY-messages.po.
First four are 'normal' messages
The fifth one correspond to an unquoted preference string
The sixth one is a normal output message from msgint command
We will hide them unless verbose '-v' option is selected,
and fix the error.
To test:
1) Go to misc/translator
2) Create new language files
$ perl translate create xx-YY
3) Check warnings and messages
4) Apply the patch
5) Remove created files, and create them again
Ther must be no messages
6) Repeat 5) with verbose option '-v', check
old messages
7) BONUS: without the patch update a language,
for example de-DE:
$ perl translate update de-DE
Note (git status) a new file, po/de-DE-messages.po~
This is a backup file from msgmerge command
With the patch this backup is removed.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Translate script ignores multiple options in system preferences,
this patch fix that.
To test:
1) Apply the patch
2) Go to misc/translator
cd misc/translator
3) Update your preferred language
perl translate update xx-YY
4) Updated pref file must contain new strings, eg.
egrep "when cataloguing an item" po/xx-YY-pref.po
5) Edit & translate that string
6) Install the updated translation
perl translate install xx-YY
7) Check that the translated pref file has
a) multiple options present
b) the message from 5) is translated
see for example circulation.pref
en: koha-tmpl/intranet-tmpl/prog/en/modules/admin/preferences/circulation.pref
xx: koha-tmpl/intranet-tmpl/prog/xx-YY/modules/admin/preferences/circulation.pref
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Mirko Tietgen <mirko@abunchofthings.net>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
The string extraction process was not taking into account the fact that
standard/package install have a completely different directory structure
than the dev install
This patch tries to keep the exact same behaviour for dev installs,
while making it work for standard install by using opachtdocs,
intrahtdocs, opacdir and intranetdir from $KOHA_CONF
Test plan:
1. Follow test plan in
d708255c7a
2. Do a standard install and repeat step 1 on this new install
3. If you know how to build the Debian package, build it, install it and
verify that koha-translate works as expected
4. prove t/LangInstaller.t
Signed-off-by: Mirko Tietgen <mirko@abunchofthings.net>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
It should make the string extraction process a little faster
For the record, I timed some parts of the process to see why it was so
slow, and without surprises the most time-consuming task is the Perl
code analysis by PPI with an average time of 50ms by "template block" on
my machine.
Multiply that by the number of template blocks (approximatively 900),
that gives us 45 seconds just for this task.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Because LangInstaller use FindBin to define some paths, the test
produces warnings about a missing directory. I'm not sure how we can
avoid that.
Test plan:
1. prove t/LangInstaller.t
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Locale::Maketext does not allow correct handling of plural translation
for languages that have more than one plural forms.
Locale::Messages does.
So Koha::I18N is now a wrapper around Locale::Messages, just like
Locale::TextDomain, and export the same symbols as Locale::TextDomain.
You can refer to documentation of Locale::TextDomain to know how to use
exported subroutines.
Example usage:
__("Hi")
__x("Hi {name}", name => 'Bob')
__n("item", "items", $num_items)
__nx("one item", "{count} items", $num_items, count => $num_items)
__p("Bibliographic record", "item")
This patch also brings Koha::I18N power to Template::Toolkit templates
by adding a TT plugin.
This plugin can be used like this:
[%# USE the plugin and define some macros %]
[% PROCESS 'i18n.inc' %]
[%# tn is the equivalent of __n %]
[%# macro names can't start with underscore, t is for "translate" %]
[% tn('item', 'items', num_items) %]
Extraction of strings from templates is a bit complicated and use
Template::Parser and PPI. Template is compiled into Perl code and then
analyzed by PPI. It is slow, but should be correct even with complex
constructions.
Remove dependency to Locale::Maketext and Locale::Maketext::Lexicon
Add dependency to Locale::Messages and PPI
Test plan for translation in Perl code:
1. Open a .pl script or .pm module with your favorite text editor
2. Add 'use Koha::I18N;' in the beginning of file
3. Use one of the subroutines exported by Koha::I18N and be sure to have
a way to visualize the result (pass result to the template for
example, or simply warn and watch the log file)
4. cd misc/translator && ./translate update fr-FR # try other languages
5. Open misc/translator/po/fr-FR-messages.po and translate your
string(s)
You may need to change the "Plural-Forms" header. See
https://localization-guide.readthedocs.org/en/latest/l10n/pluralforms.html
6. ./translate install fr-FR
7. Use your web browser to go to the page that should display the
translation, change language and verify the translation is correct
8. prove t/Koha/I18N.t
Test plan for translation in templates:
1. Open a template file (.tt or .inc) with your favorite text editor
2. Add the PROCESS directive mentioned above in the beginning of file
3. Use one of the t* macros defined in i18n.inc. They are used like
their "__" equivalent, with one difference: the 'x' variants take a
hashref instead of a hash as last parameter
4. cd misc/translator && ./translate update fr-FR
5. Open misc/translator/po/fr-FR-messages.po and translate your
string(s)
6. ./translate install fr-FR
7. Use your web browser to go to the page that should display the
translation, change language and verify the translation is
correct
Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Test plan:
1) Aply the patches
2) cd misc/translator
3) Run these command one by one:
./translate install <lang-code>
./translate update <lang-code>
./translate create <lang-code>
4) All should not end with error and must do what is documented (see
./translate --help)
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Adding one backslash makes a difference :)
We need to escape the dot in the regex to exclude a file like zzpref
from translation too. Perfect_regexes++
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Text editors can create temporary files in this folder and this can
confuse the translator.
For instance, vim can create a file named '.opac.pref.swp' which will
make the translator dies with the following error message:
Can't use string ("b0VIM 8.0") as a HASH ref while "strict refs" in use
at LangInstaller.pm line 248.
Test plan:
1. echo 'Oops' > .../en/modules/admin/preferences/whatever.pref.whatever
2. cd misc/translator && ./translate update fr-FR
3. Verify that you have the error message mentioned above
4. Apply patch
5. cd misc/translator && ./translate update fr-FR
6. No more errors!
Signed-off-by: Frédéric Demians <f.demians@tamil.fr>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Problem on this report was caused by translating the tabs Privacy
and Payments by the same string. This caused overwriting a hash entry.
This patch tests if the key already exists and if so, it merges the
entries instead of overwriting the old contents.
Test plan:
[1] Make sure that e.g. Privacy and Payments translate to e.g Vie privee.
[2] Run translate install fr-CA (or the language you altered)
[3] Without this patch you should loose preferences from either Privacy or
Payments. With this patch, they should be merged.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Tested with fr-CA.
Signed-off-by: Blou <philippe.blouin@inlibro.com>
Reset the .po files, reproduced the problem. Applied the patch and suddenly 'paypal' appeared.
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Bug 15527 add an xslt dir, LangInstaller.pm must ingnore that dir.
To test:
1) Verify the problem on current master
Update translation for any lang, will see errors
2) Apply the patch
3) Update again, no errors
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Works as expected.
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
Just upgraded our Koha container to 14.04 and noticing the
misc/translator/translate doesn't work. This is because of a missing
package gettext.
This patch makes ./translate die with a helpful suggestion to install
gettext if xgettext-program is missing.
Signed-off-by: Frederic Demians <f.demians@tamil.fr>
I can reproduce the bug by desinstalling 'gettext' package on a Debian
box. I confirm that without this patch, 'translate' command fails
miserably. Suggesting to install gettext package is welcomed.
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
This patch implements separate PO files for
different MARC dialects.
It depends on correct filenames, i.e. it will build PO
files using files with/without "unimarc|normarc|marc21"
on their names.
Changes:
A) LangInstaller.pm
Added definitions to create or update xx-YY-{MARCFLAVOR}.po,
minor change to create and install procedure, and modification
of install procedure to handle multiple target dirs.
Updated documentation.
B) Standarization of filenames
STAFF po file is now xx-YY-staff-prog.po
MARC dialects po files are xx-YY-marc-{MARCFLAVOUR}.po
To test:
1) Update po files for your preferred language, ej. nn-NO
cd misc/translator
perl translate update nn-NO
2) Do some copying/renaming
cp po/nn-NO-i-staff-t-prog-v-3006000.po po/nn-NO-marc-UNIMARC.po
cp po/nn-NO-i-staff-t-prog-v-3006000.po po/nn-NO-marc-NORMARC.po
cp po/nn-NO-i-staff-t-prog-v-3006000.po po/nn-NO-marc-MARC21.po
mv po/nn-NO-i-staff-t-prog-v-3006000.po po/nn-NO-staff-prog.po
(most MARC dialect strings are on staff, so we use that as basis)
3) Apply the patch
4) Update again to fix translation files, verbose
perl translate update nn-NO -v
5) Install language, verbose, verify translations
perl translate install nn-NO -v
6) Create translation files
rm po/nn-NO*
perl translate create nn-NO
we must have this list:
po/nn-NO-marc-MARC21.po
po/nn-NO-marc-NORMARC.po
po/nn-NO-marc-UNIMARC.po
po/nn-NO-opac-bootstrap.po
po/nn-NO-pref.po
po/nn-NO-staff-help.po
po/nn-NO-staff-prog.po
Additional tests:
7) Number of msgids
7.a) Before patch and after upgrade, extract and count msgids
for i in $(ls po/nn-NO-*po); \
do msginit -i $i -o nn-old.po --no-translator --no-wrap --locale=nn_NO; \
egrep ^msgid nn-old.po >> old; \
done
sort old | uniq | tee s-old | wc -l > n-old
s-old: have all msgids
n-old: number of msgids
7.b) After patch and after creation of new files
Repeat procedure, diferent files (s-new, n-new)
7.c) Compare (diff s-old snew), they are the same
(save for a strange UNIMARC char in my case, but
it's present on corresponding PO file)
8) Installed dirs/files
8.a) List of EN dirs/files
cd koha-tmpl
find | egrep "/en/" > en
8.b) List of nn-NO dirs/files. After patch and language install
cd koha-tmpl
find | egrep "/nn-NO/" | sed 's|/nn-NO/|/en/|' > nn
8.c) Compare (diff en nn), they are the same
Signed-off-by: Magnus Enger <magnus@enger.priv.no>
Followed the steps outlined by Bernardo, and everything works as
expected. I think the most important points are that "perl translate
create nn-NO" produces the right files, and translating anything in
them, then doing "translate install" makes the translations show
up in the interface. The numbers msgids in the nn-NO correspond
well wit the number of msgids in other sets of .po files.
I bet y'all will be happy when you don't have to see the stupid
Norwegian strings when you translate! ;-)
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
http://bugs.koha-community.org/show_bug.cgi?id=9987
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
A dir creation was uncommented (by me), the effect is
create a lang dir on 'update', which is no needed.
To test:
1) Update translation files for a uninstalled language
(cd misc/translator; ./translate update xx-YY)
2) Check new dirs
koha-tmpl/opac-tmpl/bootstrap/xx-YY/
koha-tmpl/intranet-tmpl/prog/xx-YY/
koha-tmpl/intranet-tmpl/prog/en/modules/help/xx-YY/
3) Clean all
git clean -f -d
4) Apply the patch
5) Repeat 1) (i.e. update)
6) Check no new dirs this time
Signed-off-by: Frederic Demians <f.demians@tamil.fr>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Tested according to test plan, works as described.
Passes tests and QA script.
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This patch makes opac translation files theme independent
A side effect is that current prog translation file is
renamed.
But the renaming is not added, that gives a patch of ~60M!
Current list of languages are now obtained from pref files.
To test:
1) Apply the patch
2) For your preferred language, rename opac prog file
from 'xx-YY-i-opac-t-prog-v-3006000.po' to 'xx-YY-opac-prog.po'
3) Update that language, there must be no errors
cd misc/translation
./translate update xx-YY [-v]
Check updated po files
4) Install any language, there must be no errors
./translate install xx-YY
Check prog theme for that language
5) Create translation files. Remove a language or create new lang files
./translate create zz-WW
ls -l po/zz-WW*po
check there are 3 files (and *~) for opac
6) Check updated pod for LangInstaler.pm and translate script
When PROG/CCSR themes are removed, I'll add a Bug to remove
corresponding files.
Perhaps RM could add a followup to rename all files,
cd misc/translator/po
for old in $(ls *opac-t*); do
new=$(echo $old | sed 's/-i-opac-t-prog-v-3006000/-opac-prog/');
git mv $old $new;
done
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as described, no problems found.
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This patch modifies LangInstaller.pm to catch strings
on syspref files that have no associated syspref
To test:
1) Update strings for your preffered language
2) Check that the string "you can only choose one source"
is not present on xx-YY-pref.po file
3) Apply the patch
4) Update strings again
5) Check the strig now appears [ and another one :) ]
Doing a diff before and after will show
# Enhanced Content > All
msgid "enhanced_content.pref## <strong>NOTE:</strong> you can only
choose one source of cover images from below, otherwise Koha will show
the images from all sources selected."
msgstr ""
# Local Use
msgid "local_use.pref## Nothing defined yet."
msgstr ""
6) Translate the string
7) Install your language, check Administration > System preferences >
Enhanced content > All
(on your languge) and check the string is now translated
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Instead of writing
use CGI;
use Koha::I18N;
my $cgi = new CGI;
my $lh = Koha::I18N->get_handle_from_context($cgi, 'intranet');
print $lh->maketext('my translatable text');
you can now write
use Koha::I18N;
print gettext('my translatable text');
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
CHARSET is now automatically replaced by UTF-8, and 'update' creates the
PO file if it does not exist.
Also do not try to create PO files if POT file creation failed (when
there is no messages to translate for example).
+ add some verbosity
+ add Locale::Maketext and Locale::Maketext::Lexicon to Koha
dependencies
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
You have to use the new module Koha::I18N
Code example:
use Koha::I18N;
use CGI;
my $input = new CGI;
my $lh = Koha::I18N->get_handle_from_context($input, 'intranet');
print $lh->maketext("Localized string!");
PO files are in misc/translator/po/LANG-messages.po.
Creation of PO files are integrated to existing workflow, so to create
PO file for a language, just run in misc/translator:
./translate create LANG
To update:
./translate update LANG
You can then translate the PO with your favorite editor. Strings will be
localized at runtime.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Works as advertised. Some details needing further attention noted on bug
report.
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Bug 7934 breaks the -f option introduced in bug 9161; this patch
repairs the regression.
While the generation of the command tmpl_process3.pl, a space is
missing if the -x option is given.
The tmpl_process3.pl is called like:
/home/koha/src/misc/translator/tmpl_process3.pl -q update -i
/home/koha/src/koha-tmpl/intranet-tmpl/prog/en/ -s
/home/koha/src/misc/translator/po/fr-FR-i-staff-t-prog-v-3006000.po -r
-x 'help'-f pay.tt
Revised test plan:
1) cd ./misc/translator
2) put a warn at LangInstaller.pm line 375.
3) time ./translate update fr-FR -f pay.tt
-- note the execution time and the output. The options in the
command contain "-x 'help'-f pay.tt"
The -f param is not passed to the script.
The execution time is strangely long.
5) git reset --hard origin/master
6) apply this patch
7) put a warn at LangInstaller.pm line 375.
8) time ./translate update fr-FR -f pay.tt
-- verify the output and the execution time is now corrected.
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch modifies LangInstaller.pm to enable separation
of translation for Staff UI and Staff Help files.
It's a move that make sense, Help strings accounts for
44% of total word count for Staff, and as stated on
Comment #1, it could enable a different workflow on
translation work.
It's more a hack than an elegant solution, but it works.
Feel free to suggest another approach.
To test:
We need to test complete functionality, i.e. create and update
translation files and install translation, and verify that no
string is missing.
1) Before applying the patch, we need some data from staff file.
Pick your language, say de_DE, and
cd misc/translator/
perl translate update de-DE
msginit -i po/de-DE-i-staff-t-prog-v-3006000.po -o de-old.po --no-translator --no-wrap --locale=de_DE
egrep ^msgid de-old.po | sort | tee s-old | wc -l > number-old
We have the file s-old with all strings, and the number of strings on number-old
2) Apply the patch
3) New help file is called de-DE-staff-help.po, so create one
cp'ing old staff on new help
cp po/de-DE-i-staff-t-prog-v-3006000.po po/de-DE-staff-help.po
4) Make a new update, and analize
perl translate update de-DE
msginit -i po/de-DE-i-staff-t-prog-v-3006000.po -o de-new.po --no-translator --no-wrap --locale=de_DE
msginit -i po/de-DE-staff-help.po -o de-help.po --no-translator --no-wrap --locale=de_DE
egrep ^msgid de-new.po | sort | tee s-new | wc -l > number-new
egrep ^msgid de-help.po | sort | tee s-help | wc -l > number-help
cat s-new s-help | sort | uniq | tee s-all | wc -l > number-all
cat s-new s-help | sort | uniq -d | tee s-dup | wc -l > number-dup
Ideally what we need to found is:
diff s-old s-all = zero lines (old strings vs new strings)
In my test I got one line, but it's a false positive (the string "• " is present on new staff)
On numbers,
number-old - number-new - nummber-help + number-dup = 0
or
number-old - number-all = 0
(in my test again I have 1 as result, same string. Also there are 137
repeated lines between new staff and help)
All this tells me that all string to translate are preserved
5) Install translation
perl translate install de-DE
Enable language on staff, and check that help files are translated
6) Finally, create translation files
rm po/de-DE-*
perl translate create de-DE
verify that all files are created. Tests of 4) can be repeated.
7) Verify that no strings from help are present on staff UI file
egrep help po/de-DE-i-staff-t-prog-v-3006000.po
Only results came from help-top and bottom, and a few "help" on staff strings
Signed-off-by: Fridolyn SOMERS <fridolyn.somers@biblibre.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Tested according to test plan. Checked various pages in OPAC,
staff and intranet, translation was ok.
Passes QA script and tests.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Without this patch, the -f param (for the translate script) add the
modified line in the po file BUT comment all lines from others files.
This patch concat the generated file with the previous one and merge
them. This way produce a pretty po file with all our strings :)
+ Modification in LangInstaller.pm to fix a bug when the -f param was not
given.
Test plan:
1/ Update your po file :
cd misc/translator
perl translate update fr-FR
git commit -a -m"TMP PO"
2/ Modification on 2 files:
ie.
catalogue/advsearch.tt:32 <h1>Advanced search Foo</h1>
catalogue/issuehistory.tt:38 <th>Patron Bar</th>
3/ Update your po file with only the first file:
perl translate update fr-FR -f advsearch.tt
4/ Edit your po file and check that only the foo string is present (or
git diff).
5/ Update your po file with only the second file:
perl translate update fr-FR -f issuehistory
6/ Edit your po file and check that 2 strings are present (or git diff).
7/ Change the translation for these 2 strings and delete the fuzzy
lines.
8/ Install the first file for your language:
perl translate install fr-FR -f advsearch.tt
9/ Edit your translated file and verify the string is translated. Check
that the issuehistory.tt file is not changed.
10/ Same for issuehistory:
perl translate install fr-FR -f issuehistory
11/ stash your modification in your tt files (or reset --hard):
git stash
12/ Check in your po file that the 2 strings are commented (git diff).
13/ To finish, install all template files and check that they are
replaced correctly:
perl translate install fr-FR
Signed-off-by: Frédéric Demians <f.demians@tamil.fr>
I have squashed the both follow-up patches.
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
The previous version of tmpl_process3.pl says in the pod:
-i, --input=SOURCE Get or update strings from SOURCE file.
SOURCE is a directory if -r is also specified.
But is was not possible to specify one or more files.
This patch allows us to give one or more specific file with the -f
parameter.
for eg.:
perl tmpl_process3.pl install -i /home/koha/koha-tmpl/opac-tmpl/prog/en/modules/
-o /home/koha/koha-tmpl/opac-tmpl/prog/fr-FR/modules/
-s /home/koha/misc/translator/po/fr-FR-i-opac-t-prog-v-3006000.po
-r
-f opac-account.tt
-f opac-main.tt
You can specify:
-f advsearch.tt => translate all files with a filename containing
'advsearch.tt'
or
-f search => will translate acqui/histsearch.tt, acqui/z3950_search.tt, etc.
Bug 9161: Followup: Add a -f param for the translate script
Now you can directly call the translate script
(misc/translator/translate) with the -f parameter
eg.:
./translate install|create|update -f search.tt -f main.tt
Signed-off-by: Frédéric Demians <f.demians@tamil.fr>
I've squashed the 3 patches, and reported doc into 'translate' script.
It works as advertised.
Side note: It would be great to extend this functionnality in order to
be able to apply the translation to XSL files stored outside Koha
directories hierarchy. Useful to translate site-specific XSLs defined
with XSLTResultsDisplay, and other sysprefs.
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
With this patch, header is created when creating a new syspref .po file
for a new language (translate create), and is added if it doesn't
already exist when updating an existing language (translate update).
To test:
(1) Create a new language syspref file:
./translate create -p xx-XX
Check that there is an header
(2) Update an existing syspref file without header:
./translate update fr-FR
Check that fr-FR-pref.po has a header
(3) Update an existing syspref file with header:
Modify fr-FR-pref.po. Add an email, or whatever.
./translate update fr-FR
Check that fr-FR-pref.po has a header with the manual modification
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Comment: Work as described. No errors.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
All tests and QA script pass.
Also tested that updated pref files can still be installed correctly.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Modify /misc/translator/translate script in order to manage properly alternate
OPAC templates.
To test it with new 'ccsr' template:
- Create the .po file:
./translate create fr-FR
Result: existing .po files are not modified. A new fr-FR-opac-ccsr.po file is
available.
- Install all templates :
./translate install fr-FR
Result: A new koha-tmpl/opac-tmpl/ccsr/fr-FR directory contains translated
templates.
- Update .po files:
./translate update fr-FR
Result: fr-FR .po files are update, include fr-FR-opac-ccsr.po
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
This patch deal with the -v flag that you can put on translate script.
If you run without -v, the process should be silent
if you run with -v, the process should be verbose
Signed-off-by: Frédéric Demians <f.demians@tamil.fr>
I've refactored your patch to handle verbosity directly via a translator
attribute, rather than with parameter which has to be send to each object call
method.
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
