misc/translator/translate was doing three different things:
- extract translatable strings
- create or update PO files
- install translated templates
This patch separates responsibilities by moving the string extraction
code into several 'xgettext-like' scripts and adds gulp tasks to
automate string extraction and PO files update
This has several benefits:
- gulp runs tasks in parallel, so it's a lot faster (updating all PO
files is at least 10 times faster with my 4-cores CPU)
- there is no need for $KOHA_CONF to be defined
LangInstaller.pm relied on $KOHA_CONF to get the different paths
needed. I'm not sure why, since string extraction and PO update should
work on source files, not installed files
- string extraction code can be more easily tested
This patch also brings a couple of fixes and improvements:
- TT string extraction (strings wrapped in [% t(...) %]) was done with
Template::Parser and PPI, which was extremely slow, and had some
problems (see bug 24797).
This is now done with Locale::XGettext::TT2 (new dependency) which is
a lot faster, and fixes bug 24797
- Fix header in 4 PO files
For backward compatibility, 'create' and 'update' commands of
misc/translator/translate can still be used and will execute the
corresponding gulp task
Test plan:
1. Run `yarn install` and install Locale::XGettext::TT2
2. Run `gulp po:update`
3. Verify the contents of updated PO files
4. Run `cd misc/translator && ./translate install <lang>`
5. Verify that all (templates, sysprefs, xslt, installer files) is
correctly translated
6. Run `gulp po:create --lang <lang>` and verify that it created all PO
files for that language
7. Run `prove t/misc/translator`
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Need to install yarn & gulp, no errors
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
When languages are added or removed when need to flush the caches to
keep the interface consistent
Signed-off-by: Liz Rea <wizzyrea@gmail.com>
Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch adds the ability to compress/uncompress
translations files.
On update/install the files are uncompressed first
The only gain is to use less space.
To test:
1) Apply the patch
2) Go to misc/translator
3) Try it
$ ./translate compress fr-FR (check po/*.gz)
$ ./translate uncompress fr-FR ( check normal files)
4) Try again with verbose mode
$ ./translate compress fr-FR -v (list compressed files)
$ ./translate uncompress fr-FR -v (list uncompressed files)
5) Try update compressed files
$ ./translate compress fr-FR
$ ./translate update fr-FR (result is uncompressed)
6) Try install compressed files
$ ./translate compress fr-FR
$ ./translate install fr-FR
7) Try compress all
$ ./translate compress (add '-v' for verbose output)
8) Try uncompress all
$ ./translate uncompress (add '-v' for verbose output)
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
We certainly faced 3 similar bugs due to this syntax: bug 23006, bug
22941 and bug 17526.
To prevent other issues related to this syntax this patch suggests to
replace them all in one go.
Test plan:
Confirm that the 2 syntaxes are similar
Eyeball the patch and confirm that there is no typo!
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Perl 5.26 (or earlier) introduced a security feature, where implicitly
including the program directory as a Perl library directory no longer
happens (perl -I. ).
This causes translate to fail because it cannot find the *.pm -files in
it's own directory.
This patch adds the familiar mantra
use lib $FindBin::Bin;
to the relevant scripts.
To test:
1. Install Ubuntu18.04 or something else with Perl 5.26
2. Install Koha (we use the dev-install)
3. cd $KOHA_PATH/misc/translator/
4. perl translate create fi-FI
5. Observe problems with missing modules.
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Cannot recreate the issue right now but the changes make sense.
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
http://bugs.koha-community.org/show_bug.cgi?id=9987
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This patch makes opac translation files theme independent
A side effect is that current prog translation file is
renamed.
But the renaming is not added, that gives a patch of ~60M!
Current list of languages are now obtained from pref files.
To test:
1) Apply the patch
2) For your preferred language, rename opac prog file
from 'xx-YY-i-opac-t-prog-v-3006000.po' to 'xx-YY-opac-prog.po'
3) Update that language, there must be no errors
cd misc/translation
./translate update xx-YY [-v]
Check updated po files
4) Install any language, there must be no errors
./translate install xx-YY
Check prog theme for that language
5) Create translation files. Remove a language or create new lang files
./translate create zz-WW
ls -l po/zz-WW*po
check there are 3 files (and *~) for opac
6) Check updated pod for LangInstaler.pm and translate script
When PROG/CCSR themes are removed, I'll add a Bug to remove
corresponding files.
Perhaps RM could add a followup to rename all files,
cd misc/translator/po
for old in $(ls *opac-t*); do
new=$(echo $old | sed 's/-i-opac-t-prog-v-3006000/-opac-prog/');
git mv $old $new;
done
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Works as described, no problems found.
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Using perl v5.18, the following is warned on updating/installing
templates:
Smartmatch is experimental at ./translate line 54.
Moreover, the following should exist with previous perl versions:
perlcritic xgettext
Subroutine prototypes used at line 29, column 1. See page 194 of PBP.
(Severity: 5)
Subroutine prototypes used at line 44, column 1. See page 194 of PBP.
(Severity: 5)
Subroutine prototypes used at line 60, column 1. See page 194 of PBP.
(Severity: 5)
Subroutine prototypes used at line 72, column 1. See page 194 of PBP.
(Severity: 5)
Subroutine prototypes used at line 89, column 1. See page 194 of PBP.
(Severity: 5)
Variable declared in conditional statement at line 105, column 13.
Declare variables outside of the condition. (Severity: 5)
Subroutine prototypes used at line 127, column 1. See page 194 of PBP.
(Severity: 5)
Subroutine prototypes used at line 136, column 1. See page 194 of PBP.
(Severity: 5)
Variable declared in conditional statement at line 190, column 7.
Declare variables outside of the condition. (Severity: 5)
Subroutine prototypes used at line 237, column 1. See page 194 of PBP.
(Severity: 5)
Bareword file handle opened at line 238, column 5. See pages 202,204 of
PBP. (Severity: 5)
Two-argument "open" used at line 238, column 5. See page 207 of PBP.
(Severity: 5)
Subroutine prototypes used at line 277, column 1. See page 194 of PBP.
(Severity: 5)
Subroutine prototypes used at line 311, column 1. See page 194 of PBP.
(Severity: 5)
Bareword file handle opened at line 350, column 9. See pages 202,204 of
PBP. (Severity: 5)
Two-argument "open" used at line 350, column 9. See page 207 of PBP.
(Severity: 5)
Bareword file handle opened at line 353, column 5. See pages 202,204 of
PBP. (Severity: 5)
Bareword file handle opened at line 359, column 5. See pages 202,204 of
PBP. (Severity: 5)
Two-argument "open" used at line 359, column 5. See page 207 of PBP.
(Severity: 5)
perl -wc xgettext.pl
\1 better written as $1 at TmplTokenizer.pm line 474.
main::token_negligible_p() called too early to check prototype at
xgettext.pl line 52.
xgettext.pl syntax OK
Test plan:
Apply this patch and verify the warnings/errors does not appear anymore
and no regression is found on creating/updating/installing templates.
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Work as described, no koha-qa errros.
Tested with perl v5.18.2
No warnings, no problem detected on create/update/install i18n files
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
The previous version of tmpl_process3.pl says in the pod:
-i, --input=SOURCE Get or update strings from SOURCE file.
SOURCE is a directory if -r is also specified.
But is was not possible to specify one or more files.
This patch allows us to give one or more specific file with the -f
parameter.
for eg.:
perl tmpl_process3.pl install -i /home/koha/koha-tmpl/opac-tmpl/prog/en/modules/
-o /home/koha/koha-tmpl/opac-tmpl/prog/fr-FR/modules/
-s /home/koha/misc/translator/po/fr-FR-i-opac-t-prog-v-3006000.po
-r
-f opac-account.tt
-f opac-main.tt
You can specify:
-f advsearch.tt => translate all files with a filename containing
'advsearch.tt'
or
-f search => will translate acqui/histsearch.tt, acqui/z3950_search.tt, etc.
Bug 9161: Followup: Add a -f param for the translate script
Now you can directly call the translate script
(misc/translator/translate) with the -f parameter
eg.:
./translate install|create|update -f search.tt -f main.tt
Signed-off-by: Frédéric Demians <f.demians@tamil.fr>
I've squashed the 3 patches, and reported doc into 'translate' script.
It works as advertised.
Side note: It would be great to extend this functionnality in order to
be able to apply the translation to XSL files stored outside Koha
directories hierarchy. Useful to translate site-specific XSLs defined
with XSLTResultsDisplay, and other sysprefs.
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Magnus Enger <magnus@enger.priv.no>
"perldoc misc/translator/translate" looks good and now mentions
the -v option.
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
This patch deal with the -v flag that you can put on translate script.
If you run without -v, the process should be silent
if you run with -v, the process should be verbose
Signed-off-by: Frédéric Demians <f.demians@tamil.fr>
I've refactored your patch to handle verbosity directly via a translator
attribute, rather than with parameter which has to be send to each object call
method.
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
